ml.shifu.guagua.master

Class GuaguaMasterService<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>

java.lang.Object

ml.shifu.guagua.master.GuaguaMasterService<MASTER_RESULT,WORKER_RESULT>

Type Parameters:

MASTER_RESULT - master computation result in each iteration.

WORKER_RESULT - worker computation result in each iteration.

All Implemented Interfaces:

GuaguaService

public class GuaguaMasterService<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>
extends Object
implements GuaguaService

GuaguaMasterService is the basic implementation as a master for whole guagua application process.

All the properties used to create computable instance, interceptor instances are set in props.

After masterComputable, masterInterceptors are constructed, they will be used in start(), run(Progressable), stop() to do the whole iteration logic.

GuaguaMasterService is only a skeleton and all implementations are in the interceptors and computable classes defined by user.

The execution order of preXXXX is different with postXXXX. For preXXX, the order is FIFO, but for postXXX, the order is FILO.

Constructor Summary

Constructors

Constructor and Description
GuaguaMasterService()

Method Summary

Modifier and Type	Method and Description
void	addMasterInterceptors(MasterInterceptor<MASTER_RESULT,WORKER_RESULT> masterInterceptor)
String	getAppId()
String	getContainerId()
MasterContext<MASTER_RESULT,WORKER_RESULT>	getContext()
InMemoryCoordinator<MASTER_RESULT,WORKER_RESULT>	getCoordinator()
MasterComputable<MASTER_RESULT,WORKER_RESULT>	getMasterComputable()
List<MasterInterceptor<MASTER_RESULT,WORKER_RESULT>>	getMasterInterceptors()
String	getMasterResultClassName()
double	getMinWorkersRatio()
long	getMinWorkersTimeOut()
Properties	getProps()
int	getTotalIteration()
String	getWorkerResultClassName()
int	getWorkers()
void	init(Properties props) Parameter initialization, such as set result class name, iteration number and so on.
protected MASTER_RESULT	iterate(MasterContext<MASTER_RESULT,WORKER_RESULT> context, int iteration, Progressable progress) Call each iteration computation and preIteration, postIteration in interceptors.
void	run(Progressable progress) Iterate MasterComputable.compute(MasterContext), and set hook point before and after each iteration.
void	setAppId(String appId) App id for whole application.
void	setContainerId(String containerId) Set the unique container id for master or worker.
void	setCoordinator(InMemoryCoordinator<MASTER_RESULT,WORKER_RESULT> coordinator)
void	setMasterComputable(MasterComputable<MASTER_RESULT,WORKER_RESULT> masterComputable)
void	setMasterInterceptors(List<MasterInterceptor<MASTER_RESULT,WORKER_RESULT>> masterInterceptors)
void	setMasterResultClassName(String masterResultClassName)
void	setMinWorkersRatio(double minWorkersRatio)
void	setMinWorkersTimeOut(long minWorkersTimeOut)
void	setProps(Properties props)
void	setSplits(List<GuaguaFileSplit> splits) Assign splits to each worker to make them load that data.
void	setTotalIteration(int totalIteration)
void	setWorkerResultClassName(String workerResultClassName)
void	setWorkers(int workers)
void	start() Start point for the service.
void	stop() Stop services from interceptors which is used for resource cleaning or servers shutting down.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GuaguaMasterService

public GuaguaMasterService()

Method Detail

start

public void start()

Description copied from interface: GuaguaService

Start point for the service. For example, setting up connection with zookeeper server.

Specified by:

start in interface GuaguaService

run

public void run(Progressable progress)

Iterate MasterComputable.compute(MasterContext), and set hook point before and after each iteration.

Specified by:

run in interface GuaguaService

iterate

protected MASTER_RESULT iterate(MasterContext<MASTER_RESULT,WORKER_RESULT> context,
                                int iteration,
                                Progressable progress)

Call each iteration computation and preIteration, postIteration in interceptors.

stop

public void stop()

Stop services from interceptors which is used for resource cleaning or servers shutting down.

Specified by:

stop in interface GuaguaService

init

public void init(Properties props)

Parameter initialization, such as set result class name, iteration number and so on.

Specified by:

init in interface GuaguaService

Parameters:

props - which contains different properties for master and workers to use.

setSplits

public void setSplits(List<GuaguaFileSplit> splits)

Description copied from interface: GuaguaService

Assign splits to each worker to make them load that data.

Specified by:

setSplits in interface GuaguaService

getAppId

public String getAppId()

setAppId

public void setAppId(String appId)

Description copied from interface: GuaguaService

App id for whole application. For example: Job id in MapReduce(hadoop 1.0) or application id in Yarn.

Specified by:

setAppId in interface GuaguaService

getContainerId

public String getContainerId()

setContainerId

public void setContainerId(String containerId)

Description copied from interface: GuaguaService

Set the unique container id for master or worker. For example: Task id in MapReduce or container id in Yarn.

Specified by:

setContainerId in interface GuaguaService

getContext

public MasterContext<MASTER_RESULT,WORKER_RESULT> getContext()

getMasterComputable

public MasterComputable<MASTER_RESULT,WORKER_RESULT> getMasterComputable()

setMasterComputable

public void setMasterComputable(MasterComputable<MASTER_RESULT,WORKER_RESULT> masterComputable)

getMasterInterceptors

public List<MasterInterceptor<MASTER_RESULT,WORKER_RESULT>> getMasterInterceptors()

setMasterInterceptors

public void setMasterInterceptors(List<MasterInterceptor<MASTER_RESULT,WORKER_RESULT>> masterInterceptors)

addMasterInterceptors

public void addMasterInterceptors(MasterInterceptor<MASTER_RESULT,WORKER_RESULT> masterInterceptor)

getTotalIteration

public int getTotalIteration()

setTotalIteration

public void setTotalIteration(int totalIteration)

getWorkers

public int getWorkers()

setWorkers

public void setWorkers(int workers)

getProps

public Properties getProps()

setProps

public void setProps(Properties props)

getMasterResultClassName

public String getMasterResultClassName()

setMasterResultClassName

public void setMasterResultClassName(String masterResultClassName)

getWorkerResultClassName

public String getWorkerResultClassName()

setWorkerResultClassName

public void setWorkerResultClassName(String workerResultClassName)

getMinWorkersRatio

public double getMinWorkersRatio()

getMinWorkersTimeOut

public long getMinWorkersTimeOut()

setMinWorkersRatio

public void setMinWorkersRatio(double minWorkersRatio)

setMinWorkersTimeOut

public void setMinWorkersTimeOut(long minWorkersTimeOut)

getCoordinator

public InMemoryCoordinator<MASTER_RESULT,WORKER_RESULT> getCoordinator()

setCoordinator

public void setCoordinator(InMemoryCoordinator<MASTER_RESULT,WORKER_RESULT> coordinator)