ml.shifu.guagua.unit

Class GuaguaUnitDriver<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>

java.lang.Object

ml.shifu.guagua.unit.GuaguaUnitDriver<MASTER_RESULT,WORKER_RESULT>

Type Parameters:

MASTER_RESULT - master result for computation in each iteration.

WORKER_RESULT - worker result for computation in each iteration.

public abstract class GuaguaUnitDriver<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>
extends Object

GuaguaUnitDriver is a helper class to run master, worker and intercepters in one jvm instance.

One should provide all the properties by using GuaguaUnitDriver(Properties).

Constructor Summary

Constructors

Constructor and Description
GuaguaUnitDriver(Properties props) Constructor with props setting.

Method Summary

Methods

Modifier and Type	Method and Description
protected void	doRun() Real running logic
abstract List<GuaguaFileSplit[]>	generateWorkerSplits(String inputs) Generate file splits according to inputs.
void	run() To run master-workers iteration.
protected void	setUp()
protected void	tearDown()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GuaguaUnitDriver

public GuaguaUnitDriver(Properties props)

Constructor with props setting.

To make it work, please make sure you set parameters in below:

 props.setProperty(GuaguaConstants.MASTER_COMPUTABLE_CLASS, SumMaster.class.getName());
 props.setProperty(GuaguaConstants.WORKER_COMPUTABLE_CLASS, SumWorker.class.getName());
 props.setProperty(GuaguaConstants.GUAGUA_ITERATION_COUNT, "10");
 props.setProperty(GuaguaConstants.GUAGUA_WORKER_NUMBER, "3");
 props.setProperty(GuaguaConstants.GUAGUA_MASTER_RESULT_CLASS, LongWritable.class.getName());
 props.setProperty(GuaguaConstants.GUAGUA_WORKER_RESULT_CLASS, LongWritable.class.getName());
 props.setProperty(GuaguaConstants.GUAGUA_WORKER_RESULT_CLASS, LongWritable.class.getName());
 

Method Detail

generateWorkerSplits

public abstract List<GuaguaFileSplit[]> generateWorkerSplits(String inputs)
                                                      throws IOException

Generate file splits according to inputs. It is like hadoop mapper splits.

Throws:

IOException

setUp

protected void setUp()

run

public void run()

To run master-workers iteration.

doRun

protected void doRun()

Real running logic

tearDown

protected void tearDown()