ml.shifu.guagua.master

Class AbstractMasterCoordinator<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>

java.lang.Object

ml.shifu.guagua.BasicCoordinator<MASTER_RESULT,WORKER_RESULT>

ml.shifu.guagua.master.AbstractMasterCoordinator<MASTER_RESULT,WORKER_RESULT>

Type Parameters:

MASTER_RESULT - master result for computation in each iteration.

WORKER_RESULT - worker result for computation in each iteration.

All Implemented Interfaces:

MasterInterceptor<MASTER_RESULT,WORKER_RESULT>, org.apache.zookeeper.Watcher

Direct Known Subclasses:

AsyncMasterCoordinator, NettyMasterCoordinator, SyncMasterCoordinator

public abstract class AbstractMasterCoordinator<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>
extends BasicCoordinator<MASTER_RESULT,WORKER_RESULT>
implements MasterInterceptor<MASTER_RESULT,WORKER_RESULT>

AbstractMasterCoordinator has some common implementation for both async and sync worker coordinator.

Common functions include: znodes cleaning up, fail-over support and others.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	AbstractMasterCoordinator.FailOverCommand AbstractMasterCoordinator.FailOverCommand is used to read last iteration before task failed.
protected class	AbstractMasterCoordinator.MasterElectionCommand Elect master from several backup masters.

Nested classes/interfaces inherited from class ml.shifu.guagua.BasicCoordinator

BasicCoordinator.BasicCoordinatorCommand, BasicCoordinator.CoordinatorCommand, BasicCoordinator.RetryCoordinatorCommand

Nested classes/interfaces inherited from interface org.apache.zookeeper.Watcher

org.apache.zookeeper.Watcher.Event

Field Summary

Fields inherited from class ml.shifu.guagua.BasicCoordinator

WAIT_SLOT_MILLS

Constructor Summary

Constructors

Constructor and Description
AbstractMasterCoordinator()

Method Summary

Methods

Modifier and Type	Method and Description
String	getMyBid()
protected boolean	isAllWorkersHalt(Iterable<WORKER_RESULT> workerResults) Check whether all workers are halted.
void	postApplication(MasterContext<MASTER_RESULT,WORKER_RESULT> context) The hook point after any computation logic.
void	postIteration(MasterContext<MASTER_RESULT,WORKER_RESULT> context) The hook point after computation of each iteration.
protected void	setMasterResult(MasterContext<MASTER_RESULT,WORKER_RESULT> context, String appMasterNode, String appMasterSplitNode)
void	setMyBid(String myBid)
protected void	setWorkerResults(MasterContext<MASTER_RESULT,WORKER_RESULT> context, String appCurrentWorkersNode, String appId, int iteration) Set worker results from znodes.
protected void	updateMasterHaltStatus(MasterContext<MASTER_RESULT,WORKER_RESULT> context) Check whether GuaguaConstants.GUAGUA_WORKER_HALT_ENABLE) is enabled, if yes, check whether all workers are halted and update master status.

Methods inherited from class ml.shifu.guagua.BasicCoordinator

checkAndSetZooKeeper, closeZooKeeper, getAppNode, getBaseMasterElectionNode, getBytesFromZNode, getCurrentMasterNode, getCurrentMasterSplitNode, getCurrentWorkerNode, getCurrentWorkerSplitNode, getLastMasterNode, getLastWorkerNode, getMasterBaseNode, getMasterElectionNode, getMasterNode, getMasterSerializer, getRootNode, getSleepTime, getWorkerBaseNode, getWorkerBaseNode, getWorkerNode, getWorkerSerializer, getZkConnLatch, getZooKeeper, initialize, isFixedTime, process, setBytesToZNode, setFixedTime, setMasterSerializer, setSleepTime, setWorkerSerializer, setZooKeeper, startHeartbeat, stopHeartBeat

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface ml.shifu.guagua.master.MasterInterceptor

preApplication, preIteration

Constructor Detail

AbstractMasterCoordinator

public AbstractMasterCoordinator()

Method Detail

postIteration

public void postIteration(MasterContext<MASTER_RESULT,WORKER_RESULT> context)

Description copied from interface: MasterInterceptor

The hook point after computation of each iteration.

Specified by:

postIteration in interface MasterInterceptor<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>

Parameters:

context - the master context instance which includes worker results and other useful parameters.

postApplication

public void postApplication(MasterContext<MASTER_RESULT,WORKER_RESULT> context)

Description copied from interface: MasterInterceptor

The hook point after any computation logic.

Specified by:

postApplication in interface MasterInterceptor<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>

Parameters:

context - the master context instance which includes worker results and other useful parameters.

setWorkerResults

protected void setWorkerResults(MasterContext<MASTER_RESULT,WORKER_RESULT> context,
                    String appCurrentWorkersNode,
                    String appId,
                    int iteration)
                         throws org.apache.zookeeper.KeeperException,
                                InterruptedException

Set worker results from znodes.

Throws:

org.apache.zookeeper.KeeperException

InterruptedException

updateMasterHaltStatus

protected void updateMasterHaltStatus(MasterContext<MASTER_RESULT,WORKER_RESULT> context)

Check whether GuaguaConstants.GUAGUA_WORKER_HALT_ENABLE) is enabled, if yes, check whether all workers are halted and update master status.

setMasterResult

protected void setMasterResult(MasterContext<MASTER_RESULT,WORKER_RESULT> context,
                   String appMasterNode,
                   String appMasterSplitNode)
                        throws org.apache.zookeeper.KeeperException,
                               InterruptedException

Throws:

org.apache.zookeeper.KeeperException

InterruptedException

isAllWorkersHalt

protected boolean isAllWorkersHalt(Iterable<WORKER_RESULT> workerResults)

Check whether all workers are halted.

getMyBid

public String getMyBid()

setMyBid

public void setMyBid(String myBid)