org.apache.hadoop.tools.mapred.lib

Class DynamicRecordReader<K,V>

java.lang.Object

org.apache.hadoop.mapreduce.RecordReader<K,V>

org.apache.hadoop.tools.mapred.lib.DynamicRecordReader<K,V>

All Implemented Interfaces:

Closeable, AutoCloseable

public class DynamicRecordReader<K,V>
extends org.apache.hadoop.mapreduce.RecordReader<K,V>

The DynamicRecordReader is used in conjunction with the DynamicInputFormat to implement the "Worker pattern" for DistCp. The DynamicRecordReader is responsible for: 1. Presenting the contents of each chunk to DistCp's mapper. 2. Acquiring a new chunk when the current chunk has been completely consumed, transparently.

Method Summary

Modifier and Type	Method and Description
void	close() Implementation of RecordReader::close().
K	getCurrentKey() Implementation of RecordReader::getCurrentKey().
V	getCurrentValue() Implementation of RecordReader::getCurrentValue().
float	getProgress() Implementation of RecordReader::getProgress().
void	initialize(org.apache.hadoop.mapreduce.InputSplit inputSplit, org.apache.hadoop.mapreduce.TaskAttemptContext taskAttemptContext) Implementation for RecordReader::initialize().
boolean	nextKeyValue() Implementation of RecordReader::nextValue().

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

initialize

public void initialize(org.apache.hadoop.mapreduce.InputSplit inputSplit,
                       org.apache.hadoop.mapreduce.TaskAttemptContext taskAttemptContext)
                throws IOException,
                       InterruptedException

Implementation for RecordReader::initialize(). Initializes the internal RecordReader to read from chunks.

Specified by:

initialize in class org.apache.hadoop.mapreduce.RecordReader<K,V>

Parameters:

inputSplit - The InputSplit for the map. Ignored entirely.

taskAttemptContext - The AttemptContext.

Throws:

IOException

InterruptedException

nextKeyValue

public boolean nextKeyValue()
                     throws IOException,
                            InterruptedException

Implementation of RecordReader::nextValue(). Reads the contents of the current chunk and returns them. When a chunk has been completely exhausted, an new chunk is acquired and read, transparently.

Specified by:

nextKeyValue in class org.apache.hadoop.mapreduce.RecordReader<K,V>

Returns:

True, if the nextValue() could be traversed to. False, otherwise.

Throws:

IOException

InterruptedException

getCurrentKey

public K getCurrentKey()
                throws IOException,
                       InterruptedException

Implementation of RecordReader::getCurrentKey().

Specified by:

getCurrentKey in class org.apache.hadoop.mapreduce.RecordReader<K,V>

Returns:

The key of the current record. (i.e. the source-path.)

Throws:

IOException

InterruptedException

getCurrentValue

public V getCurrentValue()
                  throws IOException,
                         InterruptedException

Implementation of RecordReader::getCurrentValue().

Specified by:

getCurrentValue in class org.apache.hadoop.mapreduce.RecordReader<K,V>

Returns:

The value of the current record. (i.e. the target-path.)

Throws:

IOException

InterruptedException

getProgress

public float getProgress()
                  throws IOException,
                         InterruptedException

Implementation of RecordReader::getProgress().

Specified by:

getProgress in class org.apache.hadoop.mapreduce.RecordReader<K,V>

Returns:

A fraction [0.0,1.0] indicating the progress of a DistCp mapper.

Throws:

IOException

InterruptedException

close

public void close()
           throws IOException

Implementation of RecordReader::close(). Closes the RecordReader.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in class org.apache.hadoop.mapreduce.RecordReader<K,V>

Throws:

IOException