ml.shifu.guagua.master
类 NettyMasterCoordinator<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>

java.lang.Object
  ml.shifu.guagua.BasicCoordinator<MASTER_RESULT,WORKER_RESULT>
      ml.shifu.guagua.master.AbstractMasterCoordinator<MASTER_RESULT,WORKER_RESULT>
          ml.shifu.guagua.master.NettyMasterCoordinator<MASTER_RESULT,WORKER_RESULT>

所有已实现的接口：

MasterInterceptor<MASTER_RESULT,WORKER_RESULT>, org.apache.zookeeper.Watcher

public class NettyMasterCoordinator<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>
extends AbstractMasterCoordinator<MASTER_RESULT,WORKER_RESULT>

A master coordinator to coordinate with workers through Netty server.

Master still updates results to Zookeeper znodes for fail-over. While workers sends results to master through Netty server connection.

Worker results are persisted into MemoryDiskList, the reason is that for big model, limited memory may not be enough to store all worker results in memory.

嵌套类摘要

从类 ml.shifu.guagua.master.AbstractMasterCoordinator 继承的嵌套类/接口

AbstractMasterCoordinator.FailOverCommand, AbstractMasterCoordinator.MasterElectionCommand

从类 ml.shifu.guagua.BasicCoordinator 继承的嵌套类/接口

BasicCoordinator.BasicCoordinatorCommand, BasicCoordinator.CoordinatorCommand, BasicCoordinator.RetryCoordinatorCommand

从接口 org.apache.zookeeper.Watcher 继承的嵌套类/接口

org.apache.zookeeper.Watcher.Event

字段摘要

从类 ml.shifu.guagua.BasicCoordinator 继承的字段

WAIT_SLOT_MILLS

构造方法摘要

NettyMasterCoordinator()

方法摘要

protected void	initialize(Properties props) Coordinator initialization.
void	postApplication(MasterContext<MASTER_RESULT,WORKER_RESULT> context) Wait for unregister message for all workers and then clean all znodes existing for this job.
void	postIteration(MasterContext<MASTER_RESULT,WORKER_RESULT> context) Update master computable result to master znode.
void	preApplication(MasterContext<MASTER_RESULT,WORKER_RESULT> context) Do initialization and fail-over checking before all iterations.
void	preIteration(MasterContext<MASTER_RESULT,WORKER_RESULT> context) Wait for all workers done in current iteration.

从类 ml.shifu.guagua.master.AbstractMasterCoordinator 继承的方法

getMyBid, isAllWorkersHalt, setMyBid, setWorkerResults, updateMasterHaltStatus

从类 ml.shifu.guagua.BasicCoordinator 继承的方法

checkAndSetZooKeeper, closeZooKeeper, getAppNode, getBaseMasterElectionNode, getBytesFromZNode, getCurrentMasterNode, getCurrentMasterSplitNode, getCurrentWorkerNode, getCurrentWorkerSplitNode, getLastMasterNode, getLastWorkerNode, getMasterBaseNode, getMasterElectionNode, getMasterNode, getMasterSerializer, getRootNode, getSleepTime, getWorkerBaseNode, getWorkerBaseNode, getWorkerNode, getWorkerSerializer, getZkConnLatch, getZooKeeper, isFixedTime, process, setBytesToZNode, setFixedTime, setMasterSerializer, setSleepTime, setWorkerSerializer, setZooKeeper, startHeartbeat, stopHeartBeat

从类 java.lang.Object 继承的方法

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造方法详细信息

NettyMasterCoordinator

public NettyMasterCoordinator()

方法详细信息

initialize

protected void initialize(Properties props)

从类 BasicCoordinator 复制的描述

Coordinator initialization.

覆盖：

类 BasicCoordinator<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable> 中的 initialize

preApplication

public void preApplication(MasterContext<MASTER_RESULT,WORKER_RESULT> context)

Do initialization and fail-over checking before all iterations.

参数：

context - the master context instance which includes worker results and other useful parameters.

preIteration

public void preIteration(MasterContext<MASTER_RESULT,WORKER_RESULT> context)

Wait for all workers done in current iteration.

参数：

context - the master context instance which includes worker results and other useful parameters.

postIteration

public void postIteration(MasterContext<MASTER_RESULT,WORKER_RESULT> context)

Update master computable result to master znode. At the same time clean znodes for old iterations. Iteration 0 and last iteration will not be removed for fail over.

指定者：

接口 MasterInterceptor<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable> 中的 postIteration

覆盖：

类 AbstractMasterCoordinator<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable> 中的 postIteration

参数：

context - the master context instance which includes worker results and other useful parameters.

postApplication

public void postApplication(MasterContext<MASTER_RESULT,WORKER_RESULT> context)

Wait for unregister message for all workers and then clean all znodes existing for this job.

指定者：

接口 MasterInterceptor<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable> 中的 postApplication

覆盖：

类 AbstractMasterCoordinator<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable> 中的 postApplication

参数：

context - the master context instance which includes worker results and other useful parameters.