Package ml.shifu.guagua.hadoop.io

This package contains hadoop related common io classes.

See: Description

Class Summary

Class	Description
GuaguaInputSplit	InputSplit implementation in guagua for Hadoop MapReduce job.
GuaguaLineRecordReader	A reader read HDFS file line by line.
GuaguaOptionsParser	Copy from hadoop GenericOptionsParser and extend it.
GuaguaSequenceAsBinaryRecordReader	A reader read HDFS sequence file key by key.
GuaguaSequenceAsTextRecordReader	A reader read HDFS sequence file key by key.
GuaguaSequenceRecordReader<KEY extends org.apache.hadoop.io.Writable,VALUE extends org.apache.hadoop.io.Writable>	A reader read HDFS sequence file key by key.
GuaguaWritableAdapter<W extends org.apache.hadoop.io.Writable>	GuaguaWritableAdapter is used to adapt hadoop io to Bytable interface.
GuaguaWritableSerializer<RESULT extends ml.shifu.guagua.io.Bytable>	GuaguaWritableSerializer is to provide the functionality to support use hadoop writable interface.

Package ml.shifu.guagua.hadoop.io Description

This package contains hadoop related common io classes.

Noticed that each class in this package is the same package and same name with the one in guagua-yarn. This is important to make sure such common hadoop io utils can be used in both guagua mapreduce and yarn implementations.

IO related classes including com.paypal.guagua.hadoop.io.GuaguaWritableAdapter and com.paypal.guagua.hadoop.io.GuaguaWritableSerializer can be used to combine hadoop Writable with guagua com.paypal.guagua.io.Bytable.

Hadoop input format and output format customization for guagua is in com.paypal.guagua.hadoop.io.GuaguaInputFormat and com.paypal.guagua.mapreduce.GuaguaOutputFormat and other related classes.