ml.shifu.guagua.hadoop.io

Class GuaguaWritableAdapter<W extends org.apache.hadoop.io.Writable>

java.lang.Object

ml.shifu.guagua.io.HaltBytable

ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter<W>

All Implemented Interfaces:

Bytable

public class GuaguaWritableAdapter<W extends org.apache.hadoop.io.Writable>
extends HaltBytable

GuaguaWritableAdapter is used to adapt hadoop io to Bytable interface.

The reason we use our own writable interface is that we'd like to not depend on hadoop io on guagua core service.

By using this adapter we don't need to implement such as IntWritable, LongWritable, Text and other common writable implementations.

Notice: If you use GuaguaWritableAdapter as your result class, you should provide the inner writable class name to GuaguaConstants.GUAGUA_MASTER_RESULT_CLASS or GuaguaConstants.GUAGUA_WORKER_RESULT_CLASS setting. That means to set '-mr' or 'wr' to hadoop writable class name. Here is a sample: hadoop jar ../lib/guagua-mapreduce-0.0.1-SNAPSHOT.jar \ ml.shifu.guagua.mapreduce.GuaguaMapReduceClient \ -libjars ../lib/guagua-mapreduce-examples-0.0.1-SNAPSHOT.jar,../lib/guava-r09-jarjar.jar,../lib/encog-core-3.1.0.jar,../lib/guagua-mapreduce-0.0.1-SNAPSHOT.jar,../lib/guagua-core-0.0.1-SNAPSHOT.jar,../lib/zookeeper-3.4.5.jar \ -i <input> \ -z <zkhost:zkport> \ -w ml.shifu.guagua.example.sum.SumWorker \ -m ml.shifu.guagua.example.sum.SumMaster \ -c 10 \ -n "Guagua Sum Master-Workers Job" \ -mr org.apache.hadoop.io.LongWritable \ -wr org.apache.hadoop.io.LongWritable \ $queue_opts

If you set your own main class, not use GuaguaYarnClient, you can set the result class like this.

 conf.set(GuaguaConstants.GUAGUA_MASTER_RESULT_CLASS, LongWritable.class.getName());
         conf.set(GuaguaConstants.GUAGUA_WORKER_RESULT_CLASS, LongWritable.class.getName());
 
 

See Also:

GuaguaWritableSerializer

Constructor Summary

Constructors

Constructor and Description
GuaguaWritableAdapter(W writable) Contructor with Hadoop Writable setting.

Method Summary

Methods

Modifier and Type	Method and Description
void	doReadFields(DataInput in)
void	doWrite(DataOutput out)
W	getWritable()
void	setWritable(W writable)
String	toString()

Methods inherited from class ml.shifu.guagua.io.HaltBytable

isHalt, readFields, setHalt, write

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

GuaguaWritableAdapter

public GuaguaWritableAdapter(W writable)

Contructor with Hadoop Writable setting.

Parameters:

writable - Hadoop Writable instance

Method Detail

doWrite

public void doWrite(DataOutput out)
             throws IOException

Specified by:

doWrite in class HaltBytable

Throws:

IOException

doReadFields

public void doReadFields(DataInput in)
                  throws IOException

Specified by:

doReadFields in class HaltBytable

Throws:

IOException

getWritable

public W getWritable()

setWritable

public void setWritable(W writable)

toString

public String toString()

Overrides:

toString in class Object