org.apache.hadoop.tools.rumen.datatypes.util

Class MapReduceJobPropertiesParser

java.lang.Object

org.apache.hadoop.tools.rumen.datatypes.util.MapReduceJobPropertiesParser

All Implemented Interfaces:

JobPropertyParser

public class MapReduceJobPropertiesParser
extends Object
implements JobPropertyParser

A default parser for MapReduce job configuration properties. MapReduce job configuration properties are represented as key-value pairs. Each key represents a configuration knob which controls or affects the behavior of a MapReduce job or a job's task. The value associated with the configuration key represents its value. Some of the keys are deprecated. As a result of deprecation some keys change or are preferred over other keys, across versions. MapReduceJobPropertiesParser is a utility class that parses MapReduce job configuration properties and converts the value into a well defined DataType. Users can use the parseJobProperty(String, String) API to process job configuration parameters. This API will parse a job property represented as a key-value pair and return the value wrapped inside a DataType. Callers can then use the returned DataType for further processing. MapReduceJobPropertiesParser thrives on the key name to decide which DataType to wrap the value with. Values for keys representing job-name, queue-name, user-name etc are wrapped inside JobName, QueueName, UserName etc respectively. Keys ending with *dir* are considered as a directory and hence gets be wrapped inside FileName. Similarly key ending with *codec*, *log*, *class* etc are also handled accordingly. Values representing basic java data-types like integer, float, double, boolean etc are wrapped inside DefaultDataType. If the key represents some jvm-level settings then only standard settings are extracted and gets wrapped inside DefaultDataType. Currently only '-Xmx' and '-Xms' settings are considered while the rest are ignored. Note that the parseJobProperty(String, String) API maps the keys to a configuration parameter listed in MRJobConfig. This not only filters non-framework specific keys thus ignoring user-specific and hard-to-parse keys but also provides a consistent view for all possible inputs. So if users invoke the parseJobProperty(String, String) API with either <"mapreduce.job.user.name", "bob"> or <"user.name", "bob">, then the result would be a UserName DataType wrapping the user-name "bob".

Constructor Summary

Constructors

Constructor and Description
MapReduceJobPropertiesParser()

Method Summary

Modifier and Type	Method and Description
static void	extractMaxHeapOpts(String javaOptions, List<String> heapOpts, List<String> others) Extracts the -Xmx heap option from the specified string.
static void	extractMinHeapOpts(String javaOptions, List<String> heapOpts, List<String> others) Extracts the -Xms heap option from the specified string.
DataType<?>	parseJobProperty(String key, String value) Parse the specified job configuration key-value pair.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MapReduceJobPropertiesParser

public MapReduceJobPropertiesParser()

Method Detail

parseJobProperty

public DataType<?> parseJobProperty(String key,
                                    String value)

Description copied from interface: JobPropertyParser

Parse the specified job configuration key-value pair.

Specified by:

parseJobProperty in interface JobPropertyParser

Returns:

Returns a DataType if this parser can parse this value. Returns 'null' otherwise.

extractMaxHeapOpts

public static void extractMaxHeapOpts(String javaOptions,
                                      List<String> heapOpts,
                                      List<String> others)

Extracts the -Xmx heap option from the specified string.

extractMinHeapOpts

public static void extractMinHeapOpts(String javaOptions,
                                      List<String> heapOpts,
                                      List<String> others)

Extracts the -Xms heap option from the specified string.