A B C D E G H I L M N P R S T W

A

abortTask(TaskAttemptContext) - 类 ml.shifu.guagua.mapreduce.GuaguaOutputCommitter 中的方法

addInputPath(Configuration, Path) - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceClient 中的静态方法

addJob(String[]) - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceClient 中的方法

Add new job to JobControl instance.

B

bytesToObject(byte[], String) - 类 ml.shifu.guagua.mapreduce.GuaguaWritableSerializer 中的方法

De-serialize from bytes to object.

BZ2 - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

C

calculateProgress(JobControl, JobClient) - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceClient 中的方法

Compute the progress of the current job submitted through the JobControl object jc to the JobClient jobClient

checkOutputSpecs(JobContext) - 类 ml.shifu.guagua.mapreduce.GuaguaOutputFormat 中的方法

cleanup(Mapper<LongWritable, Text, Text, Text>.Context) - 类 ml.shifu.guagua.mapreduce.GuaguaMapper 中的方法

cleanupJob(JobContext) - 类 ml.shifu.guagua.mapreduce.GuaguaOutputCommitter 中的方法

close() - 类 ml.shifu.guagua.mapreduce.GuaguaLineRecordReader 中的方法

close() - 类 ml.shifu.guagua.mapreduce.GuaguaMRRecordReader 中的方法

close(TaskAttemptContext) - 类 ml.shifu.guagua.mapreduce.GuaguaRecordWriter 中的方法

commitJob(JobContext) - 类 ml.shifu.guagua.mapreduce.GuaguaOutputCommitter 中的方法

commitTask(TaskAttemptContext) - 类 ml.shifu.guagua.mapreduce.GuaguaOutputCommitter 中的方法

createRecordReader(InputSplit, TaskAttemptContext) - 类 ml.shifu.guagua.mapreduce.GuaguaInputFormat 中的方法

creatJob(String[]) - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceClient 中的方法

Create Hadoop job according to arguments from main.

D

doReadFields(DataInput) - 类 ml.shifu.guagua.mapreduce.GuaguaWritableAdapter 中的方法

doWrite(DataOutput) - 类 ml.shifu.guagua.mapreduce.GuaguaWritableAdapter 中的方法

E

equals(Object) - 类 ml.shifu.guagua.mapreduce.GuaguaWritableAdapter 中的方法

G

generateWorkerSplits(String) - 类 ml.shifu.guagua.mapreduce.GuaguaMRUnitDriver 中的方法

getCombineGuaguaSplits(List<InputSplit>, long) - 类 ml.shifu.guagua.mapreduce.GuaguaInputFormat 中的静态方法

getCommandLine() - 类 ml.shifu.guagua.mapreduce.GuaguaOptionsParser 中的方法

Returns the commons-cli CommandLine object to process the parsed arguments.

getConfiguration() - 类 ml.shifu.guagua.mapreduce.GuaguaOptionsParser 中的方法

Get the modified configuration

getCurrentKey() - 类 ml.shifu.guagua.mapreduce.GuaguaLineRecordReader 中的方法

getCurrentKey() - 类 ml.shifu.guagua.mapreduce.GuaguaMRRecordReader 中的方法

This is a mock to hide Hadoop raw map iteration on map input key.

getCurrentValue() - 类 ml.shifu.guagua.mapreduce.GuaguaLineRecordReader 中的方法

getCurrentValue() - 类 ml.shifu.guagua.mapreduce.GuaguaMRRecordReader 中的方法

This is a mock to hide Hadoop raw map iteration on map input value.

getFileSplits() - 类 ml.shifu.guagua.mapreduce.GuaguaInputSplit 中的方法

getFinalCombineGuaguaSplits(List<InputSplit>, long) - 类 ml.shifu.guagua.mapreduce.GuaguaInputFormat 中的方法

Copy from pig implementation, need to check this code logic.

getGuaguaService() - 类 ml.shifu.guagua.mapreduce.GuaguaMapper 中的方法

getGuaguaSplits(JobContext) - 类 ml.shifu.guagua.mapreduce.GuaguaInputFormat 中的方法

Generate the list of files and make them into FileSplits.

getLength() - 类 ml.shifu.guagua.mapreduce.GuaguaInputSplit 中的方法

For master split, use Long.MAX_VALUE as its length to make it is the first task for Hadoop job.

getLibJars(Configuration) - 类 ml.shifu.guagua.mapreduce.GuaguaOptionsParser 中的静态方法

If libjars are set in the conf, parse the libjars.

getLocations() - 类 ml.shifu.guagua.mapreduce.GuaguaInputSplit 中的方法

This is just a mock.

getOutputCommitter(TaskAttemptContext) - 类 ml.shifu.guagua.mapreduce.GuaguaOutputFormat 中的方法

getProgress() - 类 ml.shifu.guagua.mapreduce.GuaguaLineRecordReader 中的方法

Get the progress within the split

getProgress() - 类 ml.shifu.guagua.mapreduce.GuaguaMRRecordReader 中的方法

Each iteration context.nextKeyValue should be called, and currentIteration is updated, so the progress is updated.

getRecordWriter(TaskAttemptContext) - 类 ml.shifu.guagua.mapreduce.GuaguaOutputFormat 中的方法

getRemainingArgs() - 类 ml.shifu.guagua.mapreduce.GuaguaOptionsParser 中的方法

Returns an array of Strings containing only application-specific arguments.

getSplits(JobContext) - 类 ml.shifu.guagua.mapreduce.GuaguaInputFormat 中的方法

Splitter building logic including master setting, also includes combining input feature like Pig.

getWritable() - 类 ml.shifu.guagua.mapreduce.GuaguaWritableAdapter 中的方法

GUAGUA_PROGRESS_COUNTER_GROUP_NAME - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

GUAGUA_PROGRESS_COUNTER_NAME - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

GuaguaInputFormat - ml.shifu.guagua.mapreduce 中的 类

GuaguaInputFormat is used to determine how many mappers in guagua MapReduce job.

GuaguaInputFormat() - 类 ml.shifu.guagua.mapreduce.GuaguaInputFormat 的构造方法

GuaguaInputSplit - ml.shifu.guagua.mapreduce 中的 类

InputSplit implementation in guagua for Hadoop MapReduce job.

GuaguaInputSplit() - 类 ml.shifu.guagua.mapreduce.GuaguaInputSplit 的构造方法

Default constructor without any setting.

GuaguaInputSplit(boolean, FileSplit...) - 类 ml.shifu.guagua.mapreduce.GuaguaInputSplit 的构造方法

Constructor with GuaguaInputSplit.isMaster and GuaguaInputSplit.fileSplits settings.

GuaguaInputSplit(boolean, FileSplit) - 类 ml.shifu.guagua.mapreduce.GuaguaInputSplit 的构造方法

Constructor with GuaguaInputSplit.isMaster and one FileSplit settings.

GuaguaLineRecordReader - ml.shifu.guagua.mapreduce 中的 类

Copy from LineRecordReader but to support GuaguaRecordReader interface.

GuaguaLineRecordReader() - 类 ml.shifu.guagua.mapreduce.GuaguaLineRecordReader 的构造方法

GuaguaMapper<MASTER_RESULT extends ml.shifu.guagua.io.Bytable,WORKER_RESULT extends ml.shifu.guagua.io.Bytable> - ml.shifu.guagua.mapreduce 中的 类

GuaguaMapper is the Hadoop Mapper implementation for both guagua master and guagua workers.

GuaguaMapper() - 类 ml.shifu.guagua.mapreduce.GuaguaMapper 的构造方法

GuaguaMapReduceClient - ml.shifu.guagua.mapreduce 中的 类

GuaguaMapReduceClient is the entry point for guagua mapreduce implementation application.

GuaguaMapReduceClient() - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceClient 的构造方法

Default constructor.

GuaguaMapReduceConstants - ml.shifu.guagua.mapreduce 中的 类

Constants in guagua mapreduce.

GuaguaMRRecordReader - ml.shifu.guagua.mapreduce 中的 类

GuaguaMRRecordReader is used as a mock for mapreduce reader interface, not real reading data.

GuaguaMRRecordReader() - 类 ml.shifu.guagua.mapreduce.GuaguaMRRecordReader 的构造方法

Default constructor, GuaguaMRRecordReader.totalIterations is set to default 0.

GuaguaMRRecordReader(int) - 类 ml.shifu.guagua.mapreduce.GuaguaMRRecordReader 的构造方法

Constructor with GuaguaMRRecordReader.totalIterations setting.

GuaguaMRUnitDriver<MASTER_RESULT extends ml.shifu.guagua.io.Bytable,WORKER_RESULT extends ml.shifu.guagua.io.Bytable> - ml.shifu.guagua.mapreduce 中的 类

GuaguaMRUnitDriver is used to run in-memory guagua application by using hadoop MapReduce splits.

GuaguaMRUnitDriver(Properties) - 类 ml.shifu.guagua.mapreduce.GuaguaMRUnitDriver 的构造方法

A only constructor here for local in-memory guagua job.

GuaguaOptionsParser - ml.shifu.guagua.mapreduce 中的 类

Copy from hadoop GenericOptionsParser and extend it.

GuaguaOptionsParser(Options, String[]) - 类 ml.shifu.guagua.mapreduce.GuaguaOptionsParser 的构造方法

Create an options parser with the given options to parse the args.

GuaguaOptionsParser(String[]) - 类 ml.shifu.guagua.mapreduce.GuaguaOptionsParser 的构造方法

Create an options parser to parse the args.

GuaguaOptionsParser(Configuration, String[]) - 类 ml.shifu.guagua.mapreduce.GuaguaOptionsParser 的构造方法

Create a GuaguaOptionsParser to parse only the generic Hadoop arguments.

GuaguaOptionsParser(Configuration, Options, String[]) - 类 ml.shifu.guagua.mapreduce.GuaguaOptionsParser 的构造方法

Create a GuaguaOptionsParser to parse given options as well as generic Hadoop options.

GuaguaOutputCommitter - ml.shifu.guagua.mapreduce 中的 类

This output committer doesn't do anything, meant for the case where output isn't desired, or as a base for not using FileOutputCommitter.

GuaguaOutputCommitter() - 类 ml.shifu.guagua.mapreduce.GuaguaOutputCommitter 的构造方法

GuaguaOutputFormat - ml.shifu.guagua.mapreduce 中的 类

This is for internal use only.

GuaguaOutputFormat() - 类 ml.shifu.guagua.mapreduce.GuaguaOutputFormat 的构造方法

GuaguaRecordWriter - ml.shifu.guagua.mapreduce 中的 类

We don't use GuaguaRecordWriter but hadoop MapReduce needs it.

GuaguaRecordWriter() - 类 ml.shifu.guagua.mapreduce.GuaguaRecordWriter 的构造方法

GuaguaWritableAdapter<W extends org.apache.hadoop.io.Writable> - ml.shifu.guagua.mapreduce 中的 类

GuaguaWritableAdapter is used to adapt hadoop io to Bytable interface.

GuaguaWritableAdapter(W) - 类 ml.shifu.guagua.mapreduce.GuaguaWritableAdapter 的构造方法

Contructor with Hadoop Writable setting.

GuaguaWritableSerializer<RESULT extends ml.shifu.guagua.io.Bytable> - ml.shifu.guagua.mapreduce 中的 类

GuaguaWritableSerializer is to provide the functionality to support use hadoop writable interface.

GuaguaWritableSerializer() - 类 ml.shifu.guagua.mapreduce.GuaguaWritableSerializer 的构造方法

H

HADOOP_SUCCESS - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

hashCode() - 类 ml.shifu.guagua.mapreduce.GuaguaWritableAdapter 中的方法

I

initialize(GuaguaFileSplit) - 类 ml.shifu.guagua.mapreduce.GuaguaLineRecordReader 中的方法

initialize(InputSplit, TaskAttemptContext) - 类 ml.shifu.guagua.mapreduce.GuaguaMRRecordReader 中的方法

IO_SORT_MB - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

isMaster() - 类 ml.shifu.guagua.mapreduce.GuaguaInputSplit 中的方法

isMaster() - 类 ml.shifu.guagua.mapreduce.GuaguaMapper 中的方法

isPigOrHadoopMetaFile(Path) - 类 ml.shifu.guagua.mapreduce.GuaguaInputFormat 中的方法

Whether it is not pig or hadoop meta output file.

isSplitable(JobContext, Path) - 类 ml.shifu.guagua.mapreduce.GuaguaInputFormat 中的方法

L

listStatus(Configuration, String) - 类 ml.shifu.guagua.mapreduce.GuaguaMRUnitDriver 中的方法

List input directories.

M

main(String[]) - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceClient 中的静态方法

MAPRED_CHILD_JAVA_OPTS - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

MAPRED_DEFAULT_CHILD_JAVA_OPTS - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

MAPRED_INPUT_DIR - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

MAPRED_JOB_ID - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

MAPRED_JOB_REDUCE_MEMORY_MB - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

MAPRED_MAP_MAX_ATTEMPTS - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

MAPRED_MAP_TASKS_SPECULATIVE_EXECUTION - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

MAPRED_MAX_SPLIT_SIZE - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

MAPRED_MIN_SPLIT_SIZE - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

MAPRED_REDUCE_TASKS_SPECULATIVE_EXECUTION - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

MAPRED_TASK_ID - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

MAPRED_TASK_PARTITION - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

MAPRED_TASK_TIMEOUT - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

MAPREDUCE_JOB_COUNTERS_LIMIT - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

ml.shifu.guagua.mapreduce - 软件包 ml.shifu.guagua.mapreduce

This package contains all the guagua-mapreduce implementation details based one only mapper hadoop job.

N

needsTaskCommit(TaskAttemptContext) - 类 ml.shifu.guagua.mapreduce.GuaguaOutputCommitter 中的方法

nextKeyValue() - 类 ml.shifu.guagua.mapreduce.GuaguaLineRecordReader 中的方法

nextKeyValue() - 类 ml.shifu.guagua.mapreduce.GuaguaMRRecordReader 中的方法

Update iteration number.

NUM_INPUT_FILES - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

P

PIG_HEADER - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

PIG_SCHEMA - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

printGenericCommandUsage(PrintStream) - 类 ml.shifu.guagua.mapreduce.GuaguaOptionsParser 中的静态方法

Print the usage message for generic command-line options supported.

progressOfRunningJob(ControlledJob, JobClient) - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceClient 中的方法

Returns the progress of a Job j which is part of a submitted JobControl object.

R

readFields(DataInput) - 类 ml.shifu.guagua.mapreduce.GuaguaInputSplit 中的方法

run(Mapper<LongWritable, Text, Text, Text>.Context) - 类 ml.shifu.guagua.mapreduce.GuaguaMapper 中的方法

Run guagua service according GuaguaMapper.isMaster setting.

run() - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceClient 中的方法

Run all jobs added to JobControl.

S

setCurrentIteration(int) - 类 ml.shifu.guagua.mapreduce.GuaguaMRRecordReader 中的静态方法

Should only be called in GuaguaMapper Progress callback.

setFileSplits(FileSplit[]) - 类 ml.shifu.guagua.mapreduce.GuaguaInputSplit 中的方法

setGuaguaService(GuaguaService) - 类 ml.shifu.guagua.mapreduce.GuaguaMapper 中的方法

setMaster(boolean) - 类 ml.shifu.guagua.mapreduce.GuaguaInputSplit 中的方法

setMaster(boolean) - 类 ml.shifu.guagua.mapreduce.GuaguaMapper 中的方法

setup(Mapper<LongWritable, Text, Text, Text>.Context) - 类 ml.shifu.guagua.mapreduce.GuaguaMapper 中的方法

setupJob(JobContext) - 类 ml.shifu.guagua.mapreduce.GuaguaOutputCommitter 中的方法

setupTask(TaskAttemptContext) - 类 ml.shifu.guagua.mapreduce.GuaguaOutputCommitter 中的方法

setWritable(W) - 类 ml.shifu.guagua.mapreduce.GuaguaWritableAdapter 中的方法

SPLIT_SLOP - 类 ml.shifu.guagua.mapreduce.GuaguaMapReduceConstants 中的静态变量

T

toString() - 类 ml.shifu.guagua.mapreduce.GuaguaInputSplit 中的方法

toString() - 类 ml.shifu.guagua.mapreduce.GuaguaWritableAdapter 中的方法

W

write(DataOutput) - 类 ml.shifu.guagua.mapreduce.GuaguaInputSplit 中的方法

write(Text, Text) - 类 ml.shifu.guagua.mapreduce.GuaguaRecordWriter 中的方法

A B C D E G H I L M N P R S T W