ml.shifu.guagua.hadoop.io

Class GuaguaSequenceRecordReader<KEY extends org.apache.hadoop.io.Writable,VALUE extends org.apache.hadoop.io.Writable>

java.lang.Object

ml.shifu.guagua.hadoop.io.GuaguaSequenceRecordReader<KEY,VALUE>

All Implemented Interfaces:

ml.shifu.guagua.io.GuaguaRecordReader<GuaguaWritableAdapter<KEY>,GuaguaWritableAdapter<VALUE>>

Direct Known Subclasses:

GuaguaSequenceAsBinaryRecordReader, GuaguaSequenceAsTextRecordReader

public class GuaguaSequenceRecordReader<KEY extends org.apache.hadoop.io.Writable,VALUE extends org.apache.hadoop.io.Writable>
extends Object
implements ml.shifu.guagua.io.GuaguaRecordReader<GuaguaWritableAdapter<KEY>,GuaguaWritableAdapter<VALUE>>

A reader read HDFS sequence file key by key. The types of sequence key and value are both BytesWritable.

Copy some code from SequenceFileRecordReader but to support GuaguaRecordReader interface.

If use default constructor, user should also call initialize(GuaguaFileSplit) like in below:

 this.setRecordReader(new SequenceFileRecordReader(Text.class, Text,class));
 this.getRecordReader().initialize(fileSplit);
 

or directly use other constructors:

 this.setRecordReader(new SequenceFileRecordReader(fileSplit, Text.class, Text,class));
 

Constructor Summary

Constructors

Constructor and Description
GuaguaSequenceRecordReader(Class<? extends org.apache.hadoop.io.Writable> keyClass, Class<? extends org.apache.hadoop.io.Writable> valueClass)
GuaguaSequenceRecordReader(org.apache.hadoop.conf.Configuration conf, ml.shifu.guagua.io.GuaguaFileSplit split, Class<? extends org.apache.hadoop.io.Writable> keyClass, Class<? extends org.apache.hadoop.io.Writable> valueClass)
GuaguaSequenceRecordReader(ml.shifu.guagua.io.GuaguaFileSplit split, Class<? extends org.apache.hadoop.io.Writable> keyClass, Class<? extends org.apache.hadoop.io.Writable> valueClass)

Method Summary

Modifier and Type	Method and Description
void	close()
GuaguaWritableAdapter<KEY>	getCurrentKey()
GuaguaWritableAdapter<VALUE>	getCurrentValue()
float	getProgress() Return the progress within the input split
void	initialize(ml.shifu.guagua.io.GuaguaFileSplit split)
boolean	nextKeyValue()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GuaguaSequenceRecordReader

public GuaguaSequenceRecordReader(Class<? extends org.apache.hadoop.io.Writable> keyClass,
                                  Class<? extends org.apache.hadoop.io.Writable> valueClass)
                           throws IOException

Throws:

IOException

GuaguaSequenceRecordReader

public GuaguaSequenceRecordReader(ml.shifu.guagua.io.GuaguaFileSplit split,
                                  Class<? extends org.apache.hadoop.io.Writable> keyClass,
                                  Class<? extends org.apache.hadoop.io.Writable> valueClass)
                           throws IOException

Throws:

IOException

GuaguaSequenceRecordReader

public GuaguaSequenceRecordReader(org.apache.hadoop.conf.Configuration conf,
                                  ml.shifu.guagua.io.GuaguaFileSplit split,
                                  Class<? extends org.apache.hadoop.io.Writable> keyClass,
                                  Class<? extends org.apache.hadoop.io.Writable> valueClass)
                           throws IOException

Throws:

IOException

Method Detail

getProgress

public float getProgress()
                  throws IOException

Return the progress within the input split

Returns:

0.0 to 1.0 of the input byte range

Throws:

IOException

initialize

public void initialize(ml.shifu.guagua.io.GuaguaFileSplit split)
                throws IOException

Specified by:

initialize in interface ml.shifu.guagua.io.GuaguaRecordReader<GuaguaWritableAdapter<KEY extends org.apache.hadoop.io.Writable>,GuaguaWritableAdapter<VALUE extends org.apache.hadoop.io.Writable>>

Throws:

IOException

nextKeyValue

public boolean nextKeyValue()
                     throws IOException

Specified by:

nextKeyValue in interface ml.shifu.guagua.io.GuaguaRecordReader<GuaguaWritableAdapter<KEY extends org.apache.hadoop.io.Writable>,GuaguaWritableAdapter<VALUE extends org.apache.hadoop.io.Writable>>

Throws:

IOException

getCurrentKey

public GuaguaWritableAdapter<KEY> getCurrentKey()

Specified by:

getCurrentKey in interface ml.shifu.guagua.io.GuaguaRecordReader<GuaguaWritableAdapter<KEY extends org.apache.hadoop.io.Writable>,GuaguaWritableAdapter<VALUE extends org.apache.hadoop.io.Writable>>

getCurrentValue

public GuaguaWritableAdapter<VALUE> getCurrentValue()

Specified by:

getCurrentValue in interface ml.shifu.guagua.io.GuaguaRecordReader<GuaguaWritableAdapter<KEY extends org.apache.hadoop.io.Writable>,GuaguaWritableAdapter<VALUE extends org.apache.hadoop.io.Writable>>

close

public void close()
           throws IOException

Specified by:

close in interface ml.shifu.guagua.io.GuaguaRecordReader<GuaguaWritableAdapter<KEY extends org.apache.hadoop.io.Writable>,GuaguaWritableAdapter<VALUE extends org.apache.hadoop.io.Writable>>

Throws:

IOException