ml.shifu.guagua.master

Class AsyncMasterCoordinator<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>

java.lang.Object

ml.shifu.guagua.BasicCoordinator<MASTER_RESULT,WORKER_RESULT>

ml.shifu.guagua.master.AbstractMasterCoordinator<MASTER_RESULT,WORKER_RESULT>

ml.shifu.guagua.master.AsyncMasterCoordinator<MASTER_RESULT,WORKER_RESULT>

Type Parameters:

MASTER_RESULT - master result for computation in each iteration.

WORKER_RESULT - worker result for computation in each iteration.

All Implemented Interfaces:

MasterInterceptor<MASTER_RESULT,WORKER_RESULT>, org.apache.zookeeper.Watcher

public class AsyncMasterCoordinator<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>
extends AbstractMasterCoordinator<MASTER_RESULT,WORKER_RESULT>

AsyncMasterCoordinator is used to as a barrier for each iteration.

For each iteration, AsyncMasterCoordinator will wait until all workers are done.

To start a new iteration, AsyncMasterCoordinator will write a znode for each iteration like '/_guagua/job_201312041304_189025/master/{currentIteration}' with with MasterComputable result as its data. This is like a signal to notify workers.

Workers are waiting on current master znode, if got current master znode, it will start another iteration.

Nested Class Summary

Nested classes/interfaces inherited from class ml.shifu.guagua.master.AbstractMasterCoordinator

AbstractMasterCoordinator.FailOverCommand, AbstractMasterCoordinator.MasterElectionCommand

Nested classes/interfaces inherited from class ml.shifu.guagua.BasicCoordinator

BasicCoordinator.BasicCoordinatorCommand, BasicCoordinator.CoordinatorCommand, BasicCoordinator.RetryCoordinatorCommand

Nested classes/interfaces inherited from interface org.apache.zookeeper.Watcher

org.apache.zookeeper.Watcher.Event

Field Summary

Fields

Modifier and Type	Field and Description
protected ProgressLock	workerInitLock Lock is used to check register info from all workers.
protected ProgressLock	workerIterationLock Lock is used to check iteration info from all workers.

Fields inherited from class ml.shifu.guagua.BasicCoordinator

WAIT_SLOT_MILLS

Constructor Summary

Constructors

Constructor and Description
AsyncMasterCoordinator()

Method Summary

Methods

Modifier and Type	Method and Description
String	getAppId()
int	getCurrentIteration()
void	preApplication(MasterContext<MASTER_RESULT,WORKER_RESULT> context) The hook point before any computation logic.
void	preIteration(MasterContext<MASTER_RESULT,WORKER_RESULT> context) The hook point before computation of each iteration.
void	process(org.apache.zookeeper.WatchedEvent event)
void	setAppId(String appId)
void	setCurrentIteration(int currentIteration)

Methods inherited from class ml.shifu.guagua.master.AbstractMasterCoordinator

getMyBid, isAllWorkersHalt, postApplication, postIteration, setMasterResult, setMyBid, setWorkerResults, updateMasterHaltStatus

Methods inherited from class ml.shifu.guagua.BasicCoordinator

checkAndSetZooKeeper, closeZooKeeper, getAppNode, getBaseMasterElectionNode, getBytesFromZNode, getCurrentMasterNode, getCurrentMasterSplitNode, getCurrentWorkerNode, getCurrentWorkerSplitNode, getLastMasterNode, getLastWorkerNode, getMasterBaseNode, getMasterElectionNode, getMasterNode, getMasterSerializer, getRootNode, getSleepTime, getWorkerBaseNode, getWorkerBaseNode, getWorkerNode, getWorkerSerializer, getZkConnLatch, getZooKeeper, initialize, isFixedTime, setBytesToZNode, setFixedTime, setMasterSerializer, setSleepTime, setWorkerSerializer, setZooKeeper, startHeartbeat, stopHeartBeat

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

workerInitLock

protected ProgressLock workerInitLock

Lock is used to check register info from all workers.

workerIterationLock

protected ProgressLock workerIterationLock

Lock is used to check iteration info from all workers.

Constructor Detail

AsyncMasterCoordinator

public AsyncMasterCoordinator()

Method Detail

process

public void process(org.apache.zookeeper.WatchedEvent event)

Specified by:

process in interface org.apache.zookeeper.Watcher

Overrides:

process in class BasicCoordinator<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>

getCurrentIteration

public int getCurrentIteration()

setCurrentIteration

public void setCurrentIteration(int currentIteration)

getAppId

public String getAppId()

setAppId

public void setAppId(String appId)

preApplication

public void preApplication(MasterContext<MASTER_RESULT,WORKER_RESULT> context)

Description copied from interface: MasterInterceptor

The hook point before any computation logic.

Parameters:

context - the master context instance which includes worker results and other useful parameters.

preIteration

public void preIteration(MasterContext<MASTER_RESULT,WORKER_RESULT> context)

Description copied from interface: MasterInterceptor

The hook point before computation of each iteration.

Parameters:

context - the master context instance which includes worker results and other useful parameters.