Package ml.shifu.guagua.example.lnr

Class LinearRegressionWorker

java.lang.Object

ml.shifu.guagua.worker.AbstractWorkerComputable<LinearRegressionParams,LinearRegressionParams,ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter<org.apache.hadoop.io.Text>>

ml.shifu.guagua.example.lnr.LinearRegressionWorker

All Implemented Interfaces:

ml.shifu.guagua.worker.WorkerComputable<LinearRegressionParams,LinearRegressionParams>

public class LinearRegressionWorker
extends ml.shifu.guagua.worker.AbstractWorkerComputable<LinearRegressionParams,LinearRegressionParams,ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter<org.apache.hadoop.io.Text>>

LinearRegressionWorker defines logic to accumulate local linear regression gradients.

At first iteration, wait for master to use the consistent initiating model.

At other iterations, workers include:

• 1. Update local model by using global model from last step..
• 2. Accumulate gradients by using local worker input data.
• 3. Send new local gradients to master by returning parameters.

WARNING: Input data should be normalized before, or you will get a very bad model.

Constructor Summary

Constructors

Constructor	Description
LinearRegressionWorker()

Method Summary

Modifier and Type	Method	Description
LinearRegressionParams	doCompute(ml.shifu.guagua.worker.WorkerContext<LinearRegressionParams,LinearRegressionParams> context)
void	init(ml.shifu.guagua.worker.WorkerContext<LinearRegressionParams,LinearRegressionParams> context)
void	initRecordReader(ml.shifu.guagua.io.GuaguaFileSplit fileSplit)
void	load(ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable> currentKey, ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter<org.apache.hadoop.io.Text> currentValue, ml.shifu.guagua.worker.WorkerContext<LinearRegressionParams,LinearRegressionParams> context)
protected void	postLoad(ml.shifu.guagua.worker.WorkerContext<LinearRegressionParams,LinearRegressionParams> context)

Methods inherited from class ml.shifu.guagua.worker.AbstractWorkerComputable

compute, getRecordReader, preLoad, setRecordReader

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LinearRegressionWorker

public LinearRegressionWorker()

Method Detail

initRecordReader

public void initRecordReader(ml.shifu.guagua.io.GuaguaFileSplit fileSplit)
                      throws IOException

Specified by:

initRecordReader in class ml.shifu.guagua.worker.AbstractWorkerComputable<LinearRegressionParams,LinearRegressionParams,ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter<org.apache.hadoop.io.Text>>

Throws:

IOException

init

public void init(ml.shifu.guagua.worker.WorkerContext<LinearRegressionParams,LinearRegressionParams> context)

Specified by:

init in class ml.shifu.guagua.worker.AbstractWorkerComputable<LinearRegressionParams,LinearRegressionParams,ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter<org.apache.hadoop.io.Text>>

doCompute

public LinearRegressionParams doCompute(ml.shifu.guagua.worker.WorkerContext<LinearRegressionParams,LinearRegressionParams> context)

Specified by:

doCompute in class ml.shifu.guagua.worker.AbstractWorkerComputable<LinearRegressionParams,LinearRegressionParams,ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter<org.apache.hadoop.io.Text>>

postLoad

protected void postLoad(ml.shifu.guagua.worker.WorkerContext<LinearRegressionParams,LinearRegressionParams> context)

Overrides:

postLoad in class ml.shifu.guagua.worker.AbstractWorkerComputable<LinearRegressionParams,LinearRegressionParams,ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter<org.apache.hadoop.io.Text>>

load

public void load(ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable> currentKey,
                 ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter<org.apache.hadoop.io.Text> currentValue,
                 ml.shifu.guagua.worker.WorkerContext<LinearRegressionParams,LinearRegressionParams> context)

Specified by:

load in class ml.shifu.guagua.worker.AbstractWorkerComputable<LinearRegressionParams,LinearRegressionParams,ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter<org.apache.hadoop.io.LongWritable>,ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter<org.apache.hadoop.io.Text>>