ml.shifu.guagua.yarn.example.sum

Class SumSequenceFileWorker

java.lang.Object

ml.shifu.guagua.worker.AbstractWorkerComputable<GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,GuaguaWritableAdapter<org.apache.hadoop.io.Text>,GuaguaWritableAdapter<org.apache.hadoop.io.Text>>

ml.shifu.guagua.yarn.example.sum.SumSequenceFileWorker

All Implemented Interfaces:

ml.shifu.guagua.worker.WorkerComputable<GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>>

public class SumSequenceFileWorker
extends ml.shifu.guagua.worker.AbstractWorkerComputable<GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,GuaguaWritableAdapter<org.apache.hadoop.io.Text>,GuaguaWritableAdapter<org.apache.hadoop.io.Text>>

SumSequenceFileWorker is used to accumulate the sum value for each sequence value.

Each value should be number but type is Text.

The master's sum value will be added to current iteration.

Constructor Summary

Constructors

Constructor and Description
SumSequenceFileWorker()

Method Summary

Methods

Modifier and Type	Method and Description
GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>	doCompute(ml.shifu.guagua.worker.WorkerContext<GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>> workerContext)
void	init(ml.shifu.guagua.worker.WorkerContext<GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>> workerContext)
void	initRecordReader(ml.shifu.guagua.io.GuaguaFileSplit fileSplit)
void	load(GuaguaWritableAdapter<org.apache.hadoop.io.Text> currentKey, GuaguaWritableAdapter<org.apache.hadoop.io.Text> currentValue, ml.shifu.guagua.worker.WorkerContext<GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>> workerContext)

Methods inherited from class ml.shifu.guagua.worker.AbstractWorkerComputable

compute, getRecordReader, postLoad, preLoad, setRecordReader

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SumSequenceFileWorker

public SumSequenceFileWorker()

Method Detail

init

public void init(ml.shifu.guagua.worker.WorkerContext<GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>> workerContext)

Specified by:

init in class ml.shifu.guagua.worker.AbstractWorkerComputable<GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,GuaguaWritableAdapter<org.apache.hadoop.io.Text>,GuaguaWritableAdapter<org.apache.hadoop.io.Text>>

doCompute

public GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable> doCompute(ml.shifu.guagua.worker.WorkerContext<GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>> workerContext)

Specified by:

doCompute in class ml.shifu.guagua.worker.AbstractWorkerComputable<GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,GuaguaWritableAdapter<org.apache.hadoop.io.Text>,GuaguaWritableAdapter<org.apache.hadoop.io.Text>>

load

public void load(GuaguaWritableAdapter<org.apache.hadoop.io.Text> currentKey,
        GuaguaWritableAdapter<org.apache.hadoop.io.Text> currentValue,
        ml.shifu.guagua.worker.WorkerContext<GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>> workerContext)

Specified by:

load in class ml.shifu.guagua.worker.AbstractWorkerComputable<GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,GuaguaWritableAdapter<org.apache.hadoop.io.Text>,GuaguaWritableAdapter<org.apache.hadoop.io.Text>>

initRecordReader

public void initRecordReader(ml.shifu.guagua.io.GuaguaFileSplit fileSplit)
                      throws IOException

Specified by:

initRecordReader in class ml.shifu.guagua.worker.AbstractWorkerComputable<GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,GuaguaWritableAdapter<org.apache.hadoop.io.Text>,GuaguaWritableAdapter<org.apache.hadoop.io.Text>>

Throws:

IOException