Package ml.shifu.guagua.hadoop.io

Class GuaguaWritableAdapter<W extends org.apache.hadoop.io.Writable>

  • All Implemented Interfaces:
    ml.shifu.guagua.io.Bytable
    public class GuaguaWritableAdapter<W extends org.apache.hadoop.io.Writable>
extends ml.shifu.guagua.io.HaltBytable
    GuaguaWritableAdapter is used to adapt hadoop io to Bytable interface.

    The reason we use our own writable interface is that we'd like to not depend on hadoop io on guagua core service.

    By using this adapter we don't need to implement such as IntWritable, LongWritable, Text and other common writable implementations.

    Notice: If you use GuaguaWritableAdapter as your result class, you should provide the inner writable class name to GuaguaConstants.GUAGUA_MASTER_RESULT_CLASS or GuaguaConstants.GUAGUA_WORKER_RESULT_CLASS setting. That means to set '-mr' or 'wr' to hadoop writable class name. Here is a sample: 

     
 hadoop jar ../lib/guagua-mapreduce-0.0.1-SNAPSHOT.jar \
     ml.shifu.guagua.mapreduce.GuaguaMapReduceClient   \
     -libjars ../lib/guagua-mapreduce-examples-0.0.1-SNAPSHOT.jar,../lib/guava-r09-jarjar.jar,../lib/encog-core-3.1.0.jar,../lib/guagua-mapreduce-0.0.1-SNAPSHOT.jar,../lib/guagua-core-0.0.1-SNAPSHOT.jar,../lib/zookeeper-3.4.5.jar \
     -i &lt;input&gt;  \
     -z &lt;zkhost:zkport&gt;  \
     -w ml.shifu.guagua.example.sum.SumWorker  \
     -m ml.shifu.guagua.example.sum.SumMaster  \
     -c 10 \
     -n "Guagua Sum Master-Workers Job" \
     -mr org.apache.hadoop.io.LongWritable \
     -wr org.apache.hadoop.io.LongWritable \
     $queue_opts

    If you set your own main class, not use GuaguaMapReduceClient, you can set the result class like this. 

      
         conf.set(GuaguaConstants.GUAGUA_MASTER_RESULT_CLASS, LongWritable.class.getName());
         conf.set(GuaguaConstants.GUAGUA_WORKER_RESULT_CLASS, LongWritable.class.getName());
    See Also:
    GuaguaWritableSerializer

    • Constructor Detail

      • GuaguaWritableAdapter

        public GuaguaWritableAdapter​(W writable)
        Constructor with Hadoop Writable setting.
        Parameters:
        writable - Hadoop Writable instance

    • Method Detail

      • doReadFields

        public void doReadFields​(DataInput in)
                  throws IOException
        Specified by:
        doReadFields in class ml.shifu.guagua.io.HaltBytable
        Throws:
        IOException

      • getWritable

        public W getWritable()

      • setWritable

        public void setWritable​(W writable)

      • hashCode

        public int hashCode()
        Overrides:
        hashCode in class Object