Package ml.shifu.guagua.hadoop.io
This package contains hadoop related common io classes.
Noticed that each class in this package is the same package and same name with the one in guagua-yarn. This is important to make sure such common hadoop io utils can be used in both guagua mapreduce and yarn implementations.
IO related classes including GuaguaWritableAdapter and
GuaguaWritableSerializer can be used to combine hadoop
Writable with guagua Bytable.
Hadoop input format and output format customization for guagua is in
GuaguaInputFormat and GuaguaOutputFormat and
other related classes.
-
Class Summary Class Description GuaguaInputSplit InputSplitimplementation in guagua for Hadoop MapReduce job.GuaguaLineRecordReader A reader read HDFS file line by line.GuaguaOptionsParser Copy from hadoop GenericOptionsParser and extend it.GuaguaSequenceAsBinaryRecordReader A reader read HDFS sequence file key by key.GuaguaSequenceAsTextRecordReader A reader read HDFS sequence file key by key.GuaguaSequenceRecordReader<KEY extends org.apache.hadoop.io.Writable,VALUE extends org.apache.hadoop.io.Writable> A reader read HDFS sequence file key by key.GuaguaWritableAdapter<W extends org.apache.hadoop.io.Writable> GuaguaWritableAdapteris used to adapt hadoop io toBytableinterface.GuaguaWritableSerializer<RESULT extends ml.shifu.guagua.io.Bytable> GuaguaWritableSerializeris to provide the functionality to support use hadoop writable interface.