ml.shifu.guagua.worker

Class AsyncWorkerCoordinator<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>

java.lang.Object

ml.shifu.guagua.BasicCoordinator<MASTER_RESULT,WORKER_RESULT>

ml.shifu.guagua.worker.AbstractWorkerCoordinator<MASTER_RESULT,WORKER_RESULT>

ml.shifu.guagua.worker.AsyncWorkerCoordinator<MASTER_RESULT,WORKER_RESULT>

Type Parameters:

MASTER_RESULT - master result for computation in each iteration.

WORKER_RESULT - worker result for computation in each iteration.

All Implemented Interfaces:

WorkerInterceptor<MASTER_RESULT,WORKER_RESULT>, org.apache.zookeeper.Watcher

public class AsyncWorkerCoordinator<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>
extends AbstractWorkerCoordinator<MASTER_RESULT,WORKER_RESULT>

AsyncWorkerCoordinator is used to as a worker barrier for each iteration.

For each iteration, AsyncWorkerCoordinator will wait until master's signal.

To start a new iteration, SyncMasterCoordinator will write a znode for each iteration like '/_guagua/job_201312041304_189025/master/{currentIteration}' with with MasterComputable result as its data. AsyncWorkerCoordinator is trying to detect whether it exists, if yes, to start a new iteration.

Worker result will be written into each worker iteration znode for master to get.

Nested Class Summary

Nested classes/interfaces inherited from class ml.shifu.guagua.worker.AbstractWorkerCoordinator

AbstractWorkerCoordinator.FailOverCoordinatorCommand

Nested classes/interfaces inherited from class ml.shifu.guagua.BasicCoordinator

BasicCoordinator.BasicCoordinatorCommand, BasicCoordinator.CoordinatorCommand, BasicCoordinator.GetSplitBytes, BasicCoordinator.RetryCoordinatorCommand, BasicCoordinator.SaveResultToZookeeper

Nested classes/interfaces inherited from interface org.apache.zookeeper.Watcher

org.apache.zookeeper.Watcher.Event

Field Summary

Fields

Modifier and Type	Field and Description
protected ProgressLock	masterInitLock Lock is used to check register info from master.
protected ProgressLock	masterIterationLock Lock is used to check iteration info from master.

Fields inherited from class ml.shifu.guagua.BasicCoordinator

WAIT_SLOT_MILLS

Constructor Summary

Constructors

Constructor and Description
AsyncWorkerCoordinator()

Method Summary

Modifier and Type	Method and Description
String	getAppId()
int	getCurrentIteration()
void	postIteration(WorkerContext<MASTER_RESULT,WORKER_RESULT> context) The hook point after computation of each iteration.
void	preApplication(WorkerContext<MASTER_RESULT,WORKER_RESULT> context) The hook point before any computation logic.
void	preIteration(WorkerContext<MASTER_RESULT,WORKER_RESULT> context) The hook point before computation of each iteration.
void	process(org.apache.zookeeper.WatchedEvent event)
void	setAppId(String appId)
void	setCurrentIteration(int currentIteration)

Methods inherited from class ml.shifu.guagua.worker.AbstractWorkerCoordinator

postApplication, setMasterResult

Methods inherited from class ml.shifu.guagua.BasicCoordinator

checkAndSetZooKeeper, close, getAppNode, getBaseMasterElectionNode, getBytesFromZNode, getCurrentMasterNode, getCurrentMasterSplitNode, getCurrentWorkerNode, getCurrentWorkerSplitNode, getLastMasterNode, getLastWorkerNode, getMasterBaseNode, getMasterElectionNode, getMasterNode, getMasterSerializer, getRootNode, getSleepTime, getWorkerBaseNode, getWorkerBaseNode, getWorkerNode, getWorkerSerializer, getZkConnLatch, getZooKeeper, initialize, isFixedTime, setBytesToZNode, setFixedTime, setMasterSerializer, setSleepTime, setWorkerSerializer, setZooKeeper, startHeartbeat, stopHeartBeat

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

masterInitLock

protected ProgressLock masterInitLock

Lock is used to check register info from master.

masterIterationLock

protected ProgressLock masterIterationLock

Lock is used to check iteration info from master.

Constructor Detail

AsyncWorkerCoordinator

public AsyncWorkerCoordinator()

Method Detail

process

public void process(org.apache.zookeeper.WatchedEvent event)

Specified by:

process in interface org.apache.zookeeper.Watcher

Overrides:

process in class BasicCoordinator<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>

getCurrentIteration

public int getCurrentIteration()

setCurrentIteration

public void setCurrentIteration(int currentIteration)

getAppId

public String getAppId()

setAppId

public void setAppId(String appId)

preApplication

public void preApplication(WorkerContext<MASTER_RESULT,WORKER_RESULT> context)

Description copied from interface: WorkerInterceptor

The hook point before any computation logic.

Parameters:

context - the worker context instance which includes master result and other useful parameters.

preIteration

public void preIteration(WorkerContext<MASTER_RESULT,WORKER_RESULT> context)

Description copied from interface: WorkerInterceptor

The hook point before computation of each iteration.

Specified by:

preIteration in interface WorkerInterceptor<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>

Overrides:

preIteration in class AbstractWorkerCoordinator<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>

Parameters:

context - the worker context instance which includes master result and other useful parameters.

postIteration

public void postIteration(WorkerContext<MASTER_RESULT,WORKER_RESULT> context)

Description copied from interface: WorkerInterceptor

The hook point after computation of each iteration.

Parameters:

context - the worker context instance which includes master result and other useful parameters.