org.apache.hadoop.yarn.sls.synthetic

Class SynthTraceJobProducer

java.lang.Object

org.apache.hadoop.yarn.sls.synthetic.SynthTraceJobProducer

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.tools.rumen.JobStoryProducer

public class SynthTraceJobProducer
extends Object
implements org.apache.hadoop.tools.rumen.JobStoryProducer

This is a JobStoryProducer that operates from distribution of different workloads. The .json input file is used to determine how many weight, which size, number of maps/reducers and their duration, as well as the temporal distributed of submissions. For each parameter we control avg and stdev, and generate values via normal or log-normal distributions.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SynthTraceJobProducer.JobDefinition Class used to parse a job class from file.
static class	SynthTraceJobProducer.Sample Class used to parse value sample information.
static class	SynthTraceJobProducer.TaskDefinition A task representing a type of container - e.g.
static class	SynthTraceJobProducer.TimeSample This is used to define time-varying probability of a job start-time (e.g., to simulate daily patterns).
static class	SynthTraceJobProducer.Trace Class used to parse a trace configuration file.
static class	SynthTraceJobProducer.Workload Class used to parse a workload from file.

Field Summary

Fields

Modifier and Type	Field and Description
static String	SLS_SYNTHETIC_TRACE_FILE

Constructor Summary

Constructors

Constructor and Description
SynthTraceJobProducer(org.apache.hadoop.conf.Configuration conf)
SynthTraceJobProducer(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path path)

Method Summary

Modifier and Type	Method and Description
void	close()
org.apache.hadoop.tools.rumen.JobStory	getNextJob()
int	getNodesPerRack()
int	getNumJobs()
int	getNumNodes()
long	getSeed()
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

SLS_SYNTHETIC_TRACE_FILE

public static final String SLS_SYNTHETIC_TRACE_FILE

See Also:

Constant Field Values

Constructor Detail

SynthTraceJobProducer

public SynthTraceJobProducer(org.apache.hadoop.conf.Configuration conf)
                      throws IOException

Throws:

IOException

SynthTraceJobProducer

public SynthTraceJobProducer(org.apache.hadoop.conf.Configuration conf,
                             org.apache.hadoop.fs.Path path)
                      throws IOException

Throws:

IOException

Method Detail

getNextJob

public org.apache.hadoop.tools.rumen.JobStory getNextJob()
                                                  throws IOException

Specified by:

getNextJob in interface org.apache.hadoop.tools.rumen.JobStoryProducer

Throws:

IOException

close

public void close()

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

toString

public String toString()

Overrides:

toString in class Object

getNumJobs

public int getNumJobs()

getSeed

public long getSeed()

getNodesPerRack

public int getNodesPerRack()

getNumNodes

public int getNumNodes()