org.seqdoop.hadoop_bam

Class VCFInputFormat

java.lang.Object

org.apache.hadoop.mapreduce.InputFormat<K,V>

org.apache.hadoop.mapreduce.lib.input.FileInputFormat<org.apache.hadoop.io.LongWritable,VariantContextWritable>

org.seqdoop.hadoop_bam.VCFInputFormat

public class VCFInputFormat
extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<org.apache.hadoop.io.LongWritable,VariantContextWritable>

An InputFormat for VCF files. Values are the individual records; see VCFRecordReader for the meaning of the key.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

org.apache.hadoop.mapreduce.lib.input.FileInputFormat.Counter

Field Summary

Fields

Modifier and Type	Field and Description
static String	INTERVALS_PROPERTY Filter by region, like -L in SAMtools.
static String	TRUST_EXTS_PROPERTY Whether file extensions are to be trusted, defaults to true.

Fields inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

INPUT_DIR, INPUT_DIR_RECURSIVE, NUM_INPUT_FILES, PATHFILTER_CLASS, SPLIT_MAXSIZE, SPLIT_MINSIZE

Constructor Summary

Constructors

Constructor and Description
VCFInputFormat() Creates a new input format, which will use the Configuration from the first public method called.
VCFInputFormat(org.apache.hadoop.conf.Configuration conf) Creates a new input format, reading TRUST_EXTS_PROPERTY from the given Configuration.
VCFInputFormat(Map<org.apache.hadoop.fs.Path,VCFFormat> formatMap) Creates a new input format, trusting the given Map to define the file-to-format associations.

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,VariantContextWritable>	createRecordReader(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext ctx) Returns a BCFRecordReader or VCFRecordReader as appropriate, initialized with the given parameters.
VCFFormat	getFormat(org.apache.hadoop.fs.Path path) Returns the VCFFormat corresponding to the given path.
List<org.apache.hadoop.mapreduce.InputSplit>	getSplits(org.apache.hadoop.mapreduce.JobContext job) Defers to BCFSplitGuesser as appropriate for each individual path.
protected boolean	isSplitable(org.apache.hadoop.mapreduce.JobContext context, org.apache.hadoop.fs.Path filename)
static <T extends htsjdk.samtools.util.Locatable> void	setIntervals(org.apache.hadoop.conf.Configuration conf, List<T> intervals)

Methods inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getFormatMinSplitSize, getInputDirRecursive, getInputPathFilter, getInputPaths, getMaxSplitSize, getMinSplitSize, listStatus, makeSplit, setInputDirRecursive, setInputPathFilter, setInputPaths, setInputPaths, setMaxInputSplitSize, setMinInputSplitSize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TRUST_EXTS_PROPERTY

public static final String TRUST_EXTS_PROPERTY

Whether file extensions are to be trusted, defaults to true.

See Also:

VCFFormat.inferFromFilePath(org.apache.hadoop.fs.Path), Constant Field Values

INTERVALS_PROPERTY

public static final String INTERVALS_PROPERTY

Filter by region, like -L in SAMtools. Takes a comma-separated list of intervals, e.g. chr1:1-20000,chr2:12000-20000. For programmatic use setIntervals(Configuration, List) should be preferred.

See Also:

Constant Field Values

Constructor Detail

VCFInputFormat

public VCFInputFormat()

Creates a new input format, which will use the Configuration from the first public method called. Thus this will behave as though constructed with a Configuration directly, but only after it has received it in createRecordReader (via the TaskAttemptContext) or isSplitable or getSplits (via the JobContext). Until then, other methods will throw an IllegalStateException. This constructor exists mainly as a convenience, e.g. so that VCFInputFormat can be used directly in Job.setInputFormatClass.

VCFInputFormat

public VCFInputFormat(org.apache.hadoop.conf.Configuration conf)

Creates a new input format, reading TRUST_EXTS_PROPERTY from the given Configuration.

VCFInputFormat

public VCFInputFormat(Map<org.apache.hadoop.fs.Path,VCFFormat> formatMap)

Creates a new input format, trusting the given Map to define the file-to-format associations. Neither file paths nor their contents are looked at, only the Map is used.

The Map is not copied, so it should not be modified while this input format is in use!

Method Detail

setIntervals

public static <T extends htsjdk.samtools.util.Locatable> void setIntervals(org.apache.hadoop.conf.Configuration conf,
                                                                           List<T> intervals)

getFormat

public VCFFormat getFormat(org.apache.hadoop.fs.Path path)

Returns the VCFFormat corresponding to the given path. Returns null if it cannot be determined even based on the file contents (unless future VCF/BCF formats are very different, this means that the path does not refer to a VCF or BCF file).

If this input format was constructed using a given Map<Path,VCFFormat> and the path is not contained within that map, throws an IllegalArgumentException.

isSplitable

protected boolean isSplitable(org.apache.hadoop.mapreduce.JobContext context,
                              org.apache.hadoop.fs.Path filename)

Overrides:

isSplitable in class org.apache.hadoop.mapreduce.lib.input.FileInputFormat<org.apache.hadoop.io.LongWritable,VariantContextWritable>

createRecordReader

public org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,VariantContextWritable> createRecordReader(org.apache.hadoop.mapreduce.InputSplit split,
                                                                                                                             org.apache.hadoop.mapreduce.TaskAttemptContext ctx)
                                                                                                                      throws InterruptedException,
                                                                                                                             IOException

Returns a BCFRecordReader or VCFRecordReader as appropriate, initialized with the given parameters.

Throws IllegalArgumentException if the given input split is not a FileVirtualSplit or a FileSplit, or if the path referred to is not recognized as a VCF or BCF file (see getFormat(org.apache.hadoop.fs.Path)).

Specified by:

createRecordReader in class org.apache.hadoop.mapreduce.InputFormat<org.apache.hadoop.io.LongWritable,VariantContextWritable>

Throws:

InterruptedException

IOException

getSplits

public List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext job)
                                                       throws IOException

Defers to BCFSplitGuesser as appropriate for each individual path. VCF paths do not require special handling, so their splits are left unchanged.

Overrides:

getSplits in class org.apache.hadoop.mapreduce.lib.input.FileInputFormat<org.apache.hadoop.io.LongWritable,VariantContextWritable>

Throws:

IOException