ml.shifu.guagua.worker
类 AbstractWorkerComputable<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable,KEY extends Bytable,VALUE extends Bytable>

java.lang.Object
  ml.shifu.guagua.worker.AbstractWorkerComputable<MASTER_RESULT,WORKER_RESULT,KEY,VALUE>

类型参数：

MASTER_RESULT - master result for computation in each iteration.

WORKER_RESULT - worker result for computation in each iteration.

KEY - key type for each record

VALUE - value type for each record

所有已实现的接口：

WorkerComputable<MASTER_RESULT,WORKER_RESULT>

public abstract class AbstractWorkerComputable<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable,KEY extends Bytable,VALUE extends Bytable>
extends Object
implements WorkerComputable<MASTER_RESULT,WORKER_RESULT>

Abstract WorkerComputable implementation to load data one by one and only in the very 1st iteration.

To load data successfully, make sure GuaguaRecordReader is initialized firstly.

After data is loaded in the first iteration, one can store the data into collections (meomory or disk) to do later iteration logic.

TODO how to add the first iteration to computation.

TODO add multi-thread version to load data.

构造方法摘要

AbstractWorkerComputable()

方法摘要

WORKER_RESULT	compute(WorkerContext<MASTER_RESULT,WORKER_RESULT> workerContext) Worker computation for each iteration.
abstract WORKER_RESULT	doCompute(WorkerContext<MASTER_RESULT,WORKER_RESULT> workerContext) Real computation logic after data loading.
GuaguaRecordReader<KEY,VALUE>	getRecordReader()
abstract void	init(WorkerContext<MASTER_RESULT,WORKER_RESULT> workerContext) Initialization work for the whole computation
abstract void	initRecordReader(GuaguaFileSplit fileSplit) Each GuaguaFileSplit must be initialized before loading data.
abstract void	load(KEY currentKey, VALUE currentValue, WorkerContext<MASTER_RESULT,WORKER_RESULT> workerContext) Load data one by one before computation.
protected void	postLoad(WorkerContext<MASTER_RESULT,WORKER_RESULT> workerContext) Do some post work before loading data.
protected void	preLoad(WorkerContext<MASTER_RESULT,WORKER_RESULT> workerContext) Do some pre work before loading data.
void	setRecordReader(GuaguaRecordReader<KEY,VALUE> recordReader)

从类 java.lang.Object 继承的方法

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造方法详细信息

AbstractWorkerComputable

public AbstractWorkerComputable()

方法详细信息

compute

public WORKER_RESULT compute(WorkerContext<MASTER_RESULT,WORKER_RESULT> workerContext)
                                      throws IOException

从接口 WorkerComputable 复制的描述

Worker computation for each iteration.

指定者：

接口 WorkerComputable<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable> 中的 compute

参数：

workerContext - the worker context instance which includes worker info, master result of last iteration or other useful into for each iteration.

返回：

the worker result of each iteration.

抛出：

IOException - any io exception in computation, for example, IOException in reading data.

preLoad

protected void preLoad(WorkerContext<MASTER_RESULT,WORKER_RESULT> workerContext)

Do some pre work before loading data.

postLoad

protected void postLoad(WorkerContext<MASTER_RESULT,WORKER_RESULT> workerContext)

Do some post work before loading data.

initRecordReader

public abstract void initRecordReader(GuaguaFileSplit fileSplit)
                               throws IOException

Each GuaguaFileSplit must be initialized before loading data.

抛出：

IOException

init

public abstract void init(WorkerContext<MASTER_RESULT,WORKER_RESULT> workerContext)

Initialization work for the whole computation

doCompute

public abstract WORKER_RESULT doCompute(WorkerContext<MASTER_RESULT,WORKER_RESULT> workerContext)

Real computation logic after data loading.

load

public abstract void load(KEY currentKey,
                          VALUE currentValue,
                          WorkerContext<MASTER_RESULT,WORKER_RESULT> workerContext)

Load data one by one before computation.

getRecordReader

public GuaguaRecordReader<KEY,VALUE> getRecordReader()

setRecordReader

public void setRecordReader(GuaguaRecordReader<KEY,VALUE> recordReader)