ml.shifu.guagua.worker
类 NettyWorkerCoordinator<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>

java.lang.Object
  ml.shifu.guagua.BasicCoordinator<MASTER_RESULT,WORKER_RESULT>
      ml.shifu.guagua.worker.AbstractWorkerCoordinator<MASTER_RESULT,WORKER_RESULT>
          ml.shifu.guagua.worker.NettyWorkerCoordinator<MASTER_RESULT,WORKER_RESULT>

所有已实现的接口：

WorkerInterceptor<MASTER_RESULT,WORKER_RESULT>, org.apache.zookeeper.Watcher

public class NettyWorkerCoordinator<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>
extends AbstractWorkerCoordinator<MASTER_RESULT,WORKER_RESULT>

Worker coordinator coordinates master with Netty client channel.

Worker results and iteration info is not stored in to znode like SyncWorkerCoordinator. For a big task with much more workers, this can decrease pressure on zookeeper. To leverage Netty, fast worker and master coordination is expected.

Master results are still stored into zookeeper for fail-over. Only one master per each job, this shouldn't be a burden to zookeeper.

嵌套类摘要

从类 ml.shifu.guagua.worker.AbstractWorkerCoordinator 继承的嵌套类/接口

AbstractWorkerCoordinator.FailOverCoordinatorCommand

从类 ml.shifu.guagua.BasicCoordinator 继承的嵌套类/接口

BasicCoordinator.BasicCoordinatorCommand, BasicCoordinator.CoordinatorCommand, BasicCoordinator.RetryCoordinatorCommand

从接口 org.apache.zookeeper.Watcher 继承的嵌套类/接口

org.apache.zookeeper.Watcher.Event

字段摘要

从类 ml.shifu.guagua.BasicCoordinator 继承的字段

WAIT_SLOT_MILLS

构造方法摘要

NettyWorkerCoordinator()

方法摘要

void	postApplication(WorkerContext<MASTER_RESULT,WORKER_RESULT> context) Send stop message to master and then clean resources.
void	postIteration(WorkerContext<MASTER_RESULT,WORKER_RESULT> context) Send worker results to master; wait for current master stop; get current master result.
void	preApplication(WorkerContext<MASTER_RESULT,WORKER_RESULT> context) Worker coordinator initialization.
void	preIteration(WorkerContext<MASTER_RESULT,WORKER_RESULT> context) The hook point before computation of each iteration.

从类 ml.shifu.guagua.worker.AbstractWorkerCoordinator 继承的方法

setMasterResult

从类 ml.shifu.guagua.BasicCoordinator 继承的方法

checkAndSetZooKeeper, closeZooKeeper, getAppNode, getBaseMasterElectionNode, getBytesFromZNode, getCurrentMasterNode, getCurrentMasterSplitNode, getCurrentWorkerNode, getCurrentWorkerSplitNode, getLastMasterNode, getLastWorkerNode, getMasterBaseNode, getMasterElectionNode, getMasterNode, getMasterSerializer, getRootNode, getSleepTime, getWorkerBaseNode, getWorkerBaseNode, getWorkerNode, getWorkerSerializer, getZkConnLatch, getZooKeeper, initialize, isFixedTime, process, setBytesToZNode, setFixedTime, setMasterSerializer, setSleepTime, setWorkerSerializer, setZooKeeper, startHeartbeat, stopHeartBeat

从类 java.lang.Object 继承的方法

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造方法详细信息

NettyWorkerCoordinator

public NettyWorkerCoordinator()

方法详细信息

preApplication

public void preApplication(WorkerContext<MASTER_RESULT,WORKER_RESULT> context)

Worker coordinator initialization.

• 1. Initialize Zookeeper instance to connect to zookeeper ensemble;
• 2. check fail over to recover from last failure point;
• 3. wait for master initialization and get master address from master initialization znode;
• 4. Recover last master result from master znode if fail-over task.

参数：

context - the worker context instance which includes master result and other useful parameters.

preIteration

public void preIteration(WorkerContext<MASTER_RESULT,WORKER_RESULT> context)

从接口 WorkerInterceptor 复制的描述

The hook point before computation of each iteration.

指定者：

接口 WorkerInterceptor<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable> 中的 preIteration

覆盖：

类 AbstractWorkerCoordinator<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable> 中的 preIteration

参数：

context - the worker context instance which includes master result and other useful parameters.

postIteration

public void postIteration(WorkerContext<MASTER_RESULT,WORKER_RESULT> context)

Send worker results to master; wait for current master stop; get current master result.

参数：

context - the worker context instance which includes master result and other useful parameters.

postApplication

public void postApplication(WorkerContext<MASTER_RESULT,WORKER_RESULT> context)

Send stop message to master and then clean resources.

指定者：

接口 WorkerInterceptor<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable> 中的 postApplication

覆盖：

类 AbstractWorkerCoordinator<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable> 中的 postApplication

参数：

context - the worker context instance which includes master result and other useful parameters.