Index (guagua-yarn 0.7.3 API)

A B C D E F G H I L M N P R S T W Y Z

A

addFileToClassPath(String, Configuration) - Static method in class ml.shifu.guagua.hadoop.util.HDPUtils: Add a hdfs file to classpath of one container.
addFileToResourceMap(Map<String, LocalResource>, FileSystem, Path) - Static method in class ml.shifu.guagua.yarn.util.YarnUtils: Boilerplate to add a file to the local resources..
addInputPath(Configuration, Path) - Static method in class ml.shifu.guagua.yarn.GuaguaYarnClient
addLocalClasspathToEnv(Map<String, String>, Configuration) - Static method in class ml.shifu.guagua.yarn.util.YarnUtils: Populate the environment string map to be added to the environment vars in a remote execution container.

B

bytesToObject(byte[], String) - Method in class ml.shifu.guagua.hadoop.io.GuaguaWritableSerializer: De-serialize from bytes to object.
bytesToObject(byte[], String) - Method in class ml.shifu.guagua.yarn.GuaguaWritableSerializer: Deprecated.

De-serialize from bytes to object.
BZ2 - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants

C

channelConnected(ChannelHandlerContext, ChannelStateEvent) - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask.ClientHandler
cleanup() - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask: Clean up resources used
close() - Method in class ml.shifu.guagua.hadoop.io.GuaguaLineRecordReader
close() - Method in class ml.shifu.guagua.hadoop.io.GuaguaSequenceAsBinaryRecordReader
close() - Method in class ml.shifu.guagua.hadoop.io.GuaguaSequenceAsTextRecordReader
close() - Method in class ml.shifu.guagua.hadoop.io.GuaguaSequenceRecordReader
close() - Method in class ml.shifu.guagua.yarn.GuaguaLineRecordReader: Deprecated.
close() - Method in class ml.shifu.guagua.yarn.GuaguaSequenceAsBinaryRecordReader: Deprecated.
close() - Method in class ml.shifu.guagua.yarn.GuaguaSequenceAsTextRecordReader: Deprecated.
close() - Method in class ml.shifu.guagua.yarn.GuaguaSequenceRecordReader: Deprecated.
copyLocalResourcesToFs(Configuration, ApplicationId) - Static method in class ml.shifu.guagua.yarn.util.YarnUtils: Copy batch of resources to hdfs app directory for master and containers.
copyLocalResourceToFs(String, String, Configuration, ApplicationId) - Static method in class ml.shifu.guagua.yarn.util.YarnUtils: Copy local file to a named-file in remote FS
createSplitFiles(Path, Configuration, FileSystem, List<InputSplit>) - Static method in class ml.shifu.guagua.yarn.GuaguaSplitWriter
createSplitFiles(Path, Configuration, FileSystem, T[]) - Static method in class ml.shifu.guagua.yarn.GuaguaSplitWriter
createSplitFiles(Path, Configuration, FileSystem, InputSplit[]) - Static method in class ml.shifu.guagua.yarn.GuaguaSplitWriter
CURRENT_DIR - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants

D

DEFAULT_IO_BUFFER_SIZE - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
DEFAULT_STATUS_RPC_PORT - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
DEFAULT_STATUS_RPC_SERVER_THREAD_COUNT - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
DEFAULT_TIME_OUT - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
doReadFields(DataInput) - Method in class ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter
doReadFields(DataInput) - Method in class ml.shifu.guagua.yarn.GuaguaWritableAdapter: Deprecated.
doWrite(DataOutput) - Method in class ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter
doWrite(DataOutput) - Method in class ml.shifu.guagua.yarn.GuaguaWritableAdapter: Deprecated.

E

exceptionCaught(ChannelHandlerContext, ExceptionEvent) - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask.ClientHandler
expandInputFolder(Configuration) - Static method in class ml.shifu.guagua.yarn.util.InputSplitUtils: Expand folder to all files to support all files in that folder
exportGuaguaConfiguration(Configuration, ApplicationId) - Static method in class ml.shifu.guagua.yarn.util.YarnUtils: Export our populated Configuration as an XML file to be used by the ApplicationMaster's exec container, and register it with LocalResources.

F

findContainingFile(String) - Static method in class ml.shifu.guagua.hadoop.util.HDPUtils: Find a real file that contains file name in class path.
findContainingJar(Class) - Static method in class ml.shifu.guagua.hadoop.util.HDPUtils
fromJson(String, Class<T>) - Static method in class ml.shifu.guagua.yarn.util.GsonUtils: De-serialize JSON-format string to an object.

G

generateWorkerSplits(String) - Method in class ml.shifu.guagua.hadoop.GuaguaMRUnitDriver
getAllocatedCount() - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
getAmQueue() - Method in class ml.shifu.guagua.yarn.GuaguaYarnClient
getAmRMClient() - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
getAppAttemptId() - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
getAppAttemptId() - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask
getAppDirectory(FileSystem, ApplicationId) - Static method in class ml.shifu.guagua.yarn.util.YarnUtils: Working folder to store jars, files and other resources
getAppId() - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
getAppId() - Method in class ml.shifu.guagua.yarn.GuaguaYarnClient
getAppId() - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask
getAppName() - Method in class ml.shifu.guagua.yarn.GuaguaYarnClient
getBlockIndex(BlockLocation[], long) - Static method in class ml.shifu.guagua.yarn.util.InputSplitUtils
getCombineGuaguaSplits(List<InputSplit>, long) - Static method in class ml.shifu.guagua.yarn.util.InputSplitUtils
getCommand(String, String, String, String) - Static method in class ml.shifu.guagua.yarn.util.YarnUtils: Return java command according to main class, vm args, program args and memory settings.
getCommandLine() - Method in class ml.shifu.guagua.hadoop.io.GuaguaOptionsParser: Returns the commons-cli CommandLine object to process the parsed arguments.
getCommandLine() - Method in class ml.shifu.guagua.yarn.GuaguaOptionsParser: Deprecated.

Returns the commons-cli CommandLine object to process the parsed arguments.
getCompletedCount() - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
getConfiguration() - Method in class ml.shifu.guagua.hadoop.io.GuaguaOptionsParser: Get the modified configuration
getConfiguration() - Method in class ml.shifu.guagua.yarn.GuaguaOptionsParser: Deprecated.

Get the modified configuration
getContainerArgs() - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
getContainerId() - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
getContainerId() - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask
getContainerListener() - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
getContainersToLaunch() - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
getCurrentIteration() - Method in class ml.shifu.guagua.yarn.GuaguaIterationStatus
getCurrentKey() - Method in class ml.shifu.guagua.hadoop.io.GuaguaLineRecordReader
getCurrentKey() - Method in class ml.shifu.guagua.hadoop.io.GuaguaSequenceAsBinaryRecordReader
getCurrentKey() - Method in class ml.shifu.guagua.hadoop.io.GuaguaSequenceAsTextRecordReader
getCurrentKey() - Method in class ml.shifu.guagua.hadoop.io.GuaguaSequenceRecordReader
getCurrentKey() - Method in class ml.shifu.guagua.yarn.GuaguaLineRecordReader: Deprecated.
getCurrentKey() - Method in class ml.shifu.guagua.yarn.GuaguaSequenceAsBinaryRecordReader: Deprecated.
getCurrentKey() - Method in class ml.shifu.guagua.yarn.GuaguaSequenceAsTextRecordReader: Deprecated.
getCurrentKey() - Method in class ml.shifu.guagua.yarn.GuaguaSequenceRecordReader: Deprecated.
getCurrentValue() - Method in class ml.shifu.guagua.hadoop.io.GuaguaLineRecordReader
getCurrentValue() - Method in class ml.shifu.guagua.hadoop.io.GuaguaSequenceAsBinaryRecordReader
getCurrentValue() - Method in class ml.shifu.guagua.hadoop.io.GuaguaSequenceAsTextRecordReader
getCurrentValue() - Method in class ml.shifu.guagua.hadoop.io.GuaguaSequenceRecordReader
getCurrentValue() - Method in class ml.shifu.guagua.yarn.GuaguaLineRecordReader: Deprecated.
getCurrentValue() - Method in class ml.shifu.guagua.yarn.GuaguaSequenceAsBinaryRecordReader: Deprecated.
getCurrentValue() - Method in class ml.shifu.guagua.yarn.GuaguaSequenceAsTextRecordReader: Deprecated.
getCurrentValue() - Method in class ml.shifu.guagua.yarn.GuaguaSequenceRecordReader: Deprecated.
getExecutor() - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
getFailedCount() - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
getFileSplits() - Method in class ml.shifu.guagua.hadoop.io.GuaguaInputSplit
getFileSplits() - Method in class ml.shifu.guagua.yarn.GuaguaInputSplit: Deprecated.
getFileSplits(Configuration, long) - Static method in class ml.shifu.guagua.yarn.util.InputSplitUtils: Generate the list of files and make them into FileSplits.
getFinalCombineGuaguaSplits(List<InputSplit>, long) - Static method in class ml.shifu.guagua.yarn.util.InputSplitUtils: Copy from pig implementation, need to check this code logic.
getGuaguaService() - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask
getGuaguaSplits(Configuration, long) - Static method in class ml.shifu.guagua.yarn.util.InputSplitUtils: Generate the list of files and make them into FileSplits.
getHdpVersionForHDP224() - Static method in class ml.shifu.guagua.hadoop.util.HDPUtils: A ugly method to retrieve hdp version like 2.2.4.6633 from hadoop-hdfs jar name.
getHeapPerContainer() - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
getInputSplit() - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask
getLength() - Method in class ml.shifu.guagua.hadoop.io.GuaguaInputSplit: For master split, use Long.MAX_VALUE as its length to make it is the first task for Hadoop job.
getLength() - Method in class ml.shifu.guagua.yarn.GuaguaInputSplit: Deprecated.

For master split, use Long.MAX_VALUE as its length to make it is the first task for Hadoop job.
getLibJars(Configuration) - Static method in class ml.shifu.guagua.hadoop.io.GuaguaOptionsParser: If libjars are set in the conf, parse the libjars.
getLibJars(Configuration) - Static method in class ml.shifu.guagua.yarn.GuaguaOptionsParser: Deprecated.

If libjars are set in the conf, parse the libjars.
getLocalResourceMap(Configuration, ApplicationId) - Static method in class ml.shifu.guagua.yarn.util.YarnUtils: Build local resources including main app jar, lib jars, log4j.properties and guagua-conf.xml.
getLocations() - Method in class ml.shifu.guagua.hadoop.io.GuaguaInputSplit: Data locality functions, return all hosts for all file splits.
getLocations() - Method in class ml.shifu.guagua.yarn.GuaguaInputSplit: Deprecated.

Data locality functions, return all hosts for all file splits.
getNewSplits(Configuration) - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
getNmClientAsync() - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
getPartition() - Method in class ml.shifu.guagua.yarn.GuaguaIterationStatus
getPartition() - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask
getPathForResource(FileSystem, String, ApplicationId) - Static method in class ml.shifu.guagua.yarn.util.YarnUtils: Get path to store local resources on hdfs.
getProgress() - Method in class ml.shifu.guagua.hadoop.io.GuaguaLineRecordReader: Get the progress within the split
getProgress() - Method in class ml.shifu.guagua.hadoop.io.GuaguaSequenceAsBinaryRecordReader: Return the progress within the input split
getProgress() - Method in class ml.shifu.guagua.hadoop.io.GuaguaSequenceAsTextRecordReader: Return the progress within the input split
getProgress() - Method in class ml.shifu.guagua.hadoop.io.GuaguaSequenceRecordReader: Return the progress within the input split
getProgress() - Method in class ml.shifu.guagua.yarn.GuaguaLineRecordReader: Deprecated.

Get the progress within the split
getProgress() - Method in class ml.shifu.guagua.yarn.GuaguaSequenceAsBinaryRecordReader: Deprecated.

Return the progress within the input split
getProgress() - Method in class ml.shifu.guagua.yarn.GuaguaSequenceAsTextRecordReader: Deprecated.

Return the progress within the input split
getProgress() - Method in class ml.shifu.guagua.yarn.GuaguaSequenceRecordReader: Deprecated.

Return the progress within the input split
getRemainingArgs() - Method in class ml.shifu.guagua.hadoop.io.GuaguaOptionsParser: Returns an array of Strings containing only application-specific arguments.
getRemainingArgs() - Method in class ml.shifu.guagua.yarn.GuaguaOptionsParser: Deprecated.

Returns an array of Strings containing only application-specific arguments.
getStartTime() - Method in class ml.shifu.guagua.yarn.GuaguaYarnClient
getSuccessfulCount() - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
getTime() - Method in class ml.shifu.guagua.yarn.GuaguaIterationStatus
getTotalIteration() - Method in class ml.shifu.guagua.yarn.GuaguaIterationStatus
getWritable() - Method in class ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter
getWritable() - Method in class ml.shifu.guagua.yarn.GuaguaWritableAdapter: Deprecated.
getYarnConf() - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
getYarnConf() - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask
GsonUtils - Class in ml.shifu.guagua.yarn.util: GsonUtils is used for Object-JSON JSON-Object conversion.
GUAGAU_APP_MASTER_DEFAULT_ATTMPTS - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_APP_LIBS_SEPERATOR - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_CHILD_DEFAULT_MEMORY - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_CHILD_MEMORY - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_CONF_FILE - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_HDFS_DIR - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_LOG4J_PROPERTIES - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_TASK_TIMEOUT - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_APP_JAR - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_APP_LIB_JAR - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_APP_NAME - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_CONTAINER_ARGS - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_DEFAULT_CONTAINER_JAVA_OPTS - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_DEFAULT_MASTER_MEMORY - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_DEFAULT_MAX_CONTAINER_ATTEMPTS - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_DEFAULT_PRIORITY - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_DEFAULT_QUEUE_NAME - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_INPUT_DIR - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_INPUT_SPLIT_PREFIX - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_MASTER - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_MASTER_ARGS - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_MASTER_DEFAULT_VCORES - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_MASTER_MEMORY - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_MASTER_PRIORITY - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_MASTER_VCORES - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_MAX_CONTAINER_ATTEMPTS - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_QUEUE_NAME - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_STATUS_RPC_PORT - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_TASK_DEFAULT_VCORES - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_TASK_VCORES - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
GUAGUA_YARN_TMP - Static variable in class ml.shifu.guagua.yarn.util.YarnUtils
GuaguaAppMaster - Class in ml.shifu.guagua.yarn: GuaguaAppMaster is application master to launch master and worker tasks.
GuaguaAppMaster(ContainerId, ApplicationAttemptId, Configuration) - Constructor for class ml.shifu.guagua.yarn.GuaguaAppMaster: Construct the GuaguappMaster, populate fields using env vars and set up by YARN framework in this execution container.
GuaguaInputSplit - Class in ml.shifu.guagua.hadoop.io: InputSplit implementation in guagua.
GuaguaInputSplit() - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaInputSplit: Default constructor without any setting
GuaguaInputSplit(boolean, FileSplit...) - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaInputSplit: Constructor with GuaguaInputSplit.isMaster and GuaguaInputSplit.fileSplits settings.
GuaguaInputSplit(boolean, FileSplit) - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaInputSplit: Constructor with GuaguaInputSplit.isMaster and one FileSplit settings.
GuaguaInputSplit - Class in ml.shifu.guagua.yarn: Deprecated.
use GuaguaInputSplit
GuaguaInputSplit() - Constructor for class ml.shifu.guagua.yarn.GuaguaInputSplit: Deprecated.

Default constructor without any setting
GuaguaInputSplit(boolean, FileSplit...) - Constructor for class ml.shifu.guagua.yarn.GuaguaInputSplit: Deprecated.

Constructor with GuaguaInputSplit.isMaster and GuaguaInputSplit.fileSplits settings.
GuaguaInputSplit(boolean, FileSplit) - Constructor for class ml.shifu.guagua.yarn.GuaguaInputSplit: Deprecated.

Constructor with GuaguaInputSplit.isMaster and one FileSplit settings.
GuaguaIterationStatus - Class in ml.shifu.guagua.yarn: GuaguaIterationStatus is used to send iteration status from master and worker to Guagua application master.
GuaguaIterationStatus() - Constructor for class ml.shifu.guagua.yarn.GuaguaIterationStatus: Default constructor.
GuaguaIterationStatus(int, int, int) - Constructor for class ml.shifu.guagua.yarn.GuaguaIterationStatus: Constructor with partition, current iteration and total iteration setting.
GuaguaLineRecordReader - Class in ml.shifu.guagua.hadoop.io: Copy from LineRecordReader to avoid using Mapreduce-related interfaces.
GuaguaLineRecordReader() - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaLineRecordReader
GuaguaLineRecordReader(byte[]) - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaLineRecordReader
GuaguaLineRecordReader(GuaguaFileSplit) - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaLineRecordReader
GuaguaLineRecordReader(Configuration, GuaguaFileSplit) - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaLineRecordReader
GuaguaLineRecordReader - Class in ml.shifu.guagua.yarn: Deprecated.
use GuaguaLineRecordReader
GuaguaLineRecordReader() - Constructor for class ml.shifu.guagua.yarn.GuaguaLineRecordReader: Deprecated.
GuaguaLineRecordReader(byte[]) - Constructor for class ml.shifu.guagua.yarn.GuaguaLineRecordReader: Deprecated.
GuaguaMRUnitDriver<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable> - Class in ml.shifu.guagua.hadoop: GuaguaMRUnitDriver is used to run in-memory guagua application by using hadoop MapReduce splits.
GuaguaMRUnitDriver(Properties) - Constructor for class ml.shifu.guagua.hadoop.GuaguaMRUnitDriver: A only constructor here for local in-memory guagua job.
GuaguaOptionsParser - Class in ml.shifu.guagua.hadoop.io: GuaguaOptionsParser is a utility to parse command line arguments generic to the Hadoop framework.
GuaguaOptionsParser(Options, String[]) - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaOptionsParser: Create an options parser with the given options to parse the args.
GuaguaOptionsParser(String[]) - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaOptionsParser: Create an options parser to parse the args.
GuaguaOptionsParser(Configuration, String[]) - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaOptionsParser: Create a GuaguaOptionsParser to parse only the generic Hadoop arguments.
GuaguaOptionsParser(Configuration, Options, String[]) - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaOptionsParser: Create a GuaguaOptionsParser to parse given options as well as generic Hadoop options.
GuaguaOptionsParser - Class in ml.shifu.guagua.yarn: Deprecated.
use GuaguaOptionsParser
GuaguaOptionsParser(Options, String[]) - Constructor for class ml.shifu.guagua.yarn.GuaguaOptionsParser: Deprecated.

Create an options parser with the given options to parse the args.
GuaguaOptionsParser(String[]) - Constructor for class ml.shifu.guagua.yarn.GuaguaOptionsParser: Deprecated.

Create an options parser to parse the args.
GuaguaOptionsParser(Configuration, String[]) - Constructor for class ml.shifu.guagua.yarn.GuaguaOptionsParser: Deprecated.

Create a GuaguaOptionsParser to parse only the generic Hadoop arguments.
GuaguaOptionsParser(Configuration, Options, String[]) - Constructor for class ml.shifu.guagua.yarn.GuaguaOptionsParser: Deprecated.

Create a GuaguaOptionsParser to parse given options as well as generic Hadoop options.
GuaguaSequenceAsBinaryRecordReader - Class in ml.shifu.guagua.hadoop.io: A reader read HDFS sequence file key by key.
GuaguaSequenceAsBinaryRecordReader() - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaSequenceAsBinaryRecordReader
GuaguaSequenceAsBinaryRecordReader(GuaguaFileSplit) - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaSequenceAsBinaryRecordReader
GuaguaSequenceAsBinaryRecordReader(Configuration, GuaguaFileSplit) - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaSequenceAsBinaryRecordReader
GuaguaSequenceAsBinaryRecordReader - Class in ml.shifu.guagua.yarn: Deprecated.
use GuaguaSequenceAsBinaryRecordReader
GuaguaSequenceAsBinaryRecordReader() - Constructor for class ml.shifu.guagua.yarn.GuaguaSequenceAsBinaryRecordReader: Deprecated.
GuaguaSequenceAsBinaryRecordReader(GuaguaFileSplit) - Constructor for class ml.shifu.guagua.yarn.GuaguaSequenceAsBinaryRecordReader: Deprecated.
GuaguaSequenceAsBinaryRecordReader(Configuration, GuaguaFileSplit) - Constructor for class ml.shifu.guagua.yarn.GuaguaSequenceAsBinaryRecordReader: Deprecated.
GuaguaSequenceAsTextRecordReader - Class in ml.shifu.guagua.hadoop.io: A reader read HDFS sequence file key by key.
GuaguaSequenceAsTextRecordReader() - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaSequenceAsTextRecordReader
GuaguaSequenceAsTextRecordReader(GuaguaFileSplit) - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaSequenceAsTextRecordReader
GuaguaSequenceAsTextRecordReader(Configuration, GuaguaFileSplit) - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaSequenceAsTextRecordReader
GuaguaSequenceAsTextRecordReader - Class in ml.shifu.guagua.yarn: Deprecated.
use GuaguaSequenceAsTextRecordReader
GuaguaSequenceAsTextRecordReader() - Constructor for class ml.shifu.guagua.yarn.GuaguaSequenceAsTextRecordReader: Deprecated.
GuaguaSequenceAsTextRecordReader(GuaguaFileSplit) - Constructor for class ml.shifu.guagua.yarn.GuaguaSequenceAsTextRecordReader: Deprecated.
GuaguaSequenceAsTextRecordReader(Configuration, GuaguaFileSplit) - Constructor for class ml.shifu.guagua.yarn.GuaguaSequenceAsTextRecordReader: Deprecated.
GuaguaSequenceRecordReader<KEY extends org.apache.hadoop.io.Writable,VALUE extends org.apache.hadoop.io.Writable> - Class in ml.shifu.guagua.hadoop.io: A reader read HDFS sequence file key by key.
GuaguaSequenceRecordReader(Class<? extends Writable>, Class<? extends Writable>) - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaSequenceRecordReader
GuaguaSequenceRecordReader(GuaguaFileSplit, Class<? extends Writable>, Class<? extends Writable>) - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaSequenceRecordReader
GuaguaSequenceRecordReader(Configuration, GuaguaFileSplit, Class<? extends Writable>, Class<? extends Writable>) - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaSequenceRecordReader
GuaguaSequenceRecordReader<KEY extends org.apache.hadoop.io.Writable,VALUE extends org.apache.hadoop.io.Writable> - Class in ml.shifu.guagua.yarn: Deprecated.
use GuaguaSequenceRecordReader
GuaguaSequenceRecordReader(Class<? extends Writable>, Class<? extends Writable>) - Constructor for class ml.shifu.guagua.yarn.GuaguaSequenceRecordReader: Deprecated.
GuaguaSequenceRecordReader(GuaguaFileSplit, Class<? extends Writable>, Class<? extends Writable>) - Constructor for class ml.shifu.guagua.yarn.GuaguaSequenceRecordReader: Deprecated.
GuaguaSequenceRecordReader(Configuration, GuaguaFileSplit, Class<? extends Writable>, Class<? extends Writable>) - Constructor for class ml.shifu.guagua.yarn.GuaguaSequenceRecordReader: Deprecated.
GuaguaSplitWriter - Class in ml.shifu.guagua.yarn: The class that is used by the Job clients to write splits (both the meta and the raw bytes parts)
GuaguaSplitWriter() - Constructor for class ml.shifu.guagua.yarn.GuaguaSplitWriter
GuaguaWritableAdapter<W extends org.apache.hadoop.io.Writable> - Class in ml.shifu.guagua.hadoop.io: GuaguaWritableAdapter is used to adapt hadoop io to Bytable interface.
GuaguaWritableAdapter(W) - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter: Contructor with Hadoop Writable setting.
GuaguaWritableAdapter<W extends org.apache.hadoop.io.Writable> - Class in ml.shifu.guagua.yarn: Deprecated.
use GuaguaWritableAdapter
GuaguaWritableAdapter(W) - Constructor for class ml.shifu.guagua.yarn.GuaguaWritableAdapter: Deprecated.

Contructor with Hadoop Writable setting.
GuaguaWritableSerializer<RESULT extends Bytable> - Class in ml.shifu.guagua.hadoop.io: GuaguaWritableSerializer is to provide the functionality to support use hadoop writable interface.
GuaguaWritableSerializer() - Constructor for class ml.shifu.guagua.hadoop.io.GuaguaWritableSerializer
GuaguaWritableSerializer<RESULT extends Bytable> - Class in ml.shifu.guagua.yarn: Deprecated.
use GuaguaWritableSerializer
GuaguaWritableSerializer() - Constructor for class ml.shifu.guagua.yarn.GuaguaWritableSerializer: Deprecated.
GuaguaYarnClient - Class in ml.shifu.guagua.yarn: GuaguaYarnClient is used to submit master-workers computation app on yarn cluster.
GuaguaYarnClient() - Constructor for class ml.shifu.guagua.yarn.GuaguaYarnClient: Default constructor.
GuaguaYarnClient(Configuration) - Constructor for class ml.shifu.guagua.yarn.GuaguaYarnClient: Constructor with Configuration setting.
GuaguaYarnConstants - Class in ml.shifu.guagua.yarn: Constants in guagua yarn.
GuaguaYarnTask<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable> - Class in ml.shifu.guagua.yarn: GuaguaYarnTask is a entry point to run both master and workers.
GuaguaYarnTask(ApplicationAttemptId, ContainerId, int, String, String, Configuration) - Constructor for class ml.shifu.guagua.yarn.GuaguaYarnTask: Constructor with yarn task related parameters.
GuaguaYarnTask.ClientHandler - Class in ml.shifu.guagua.yarn: ClientHandeler used to update progress to RPC server (AppMaster).
GuaguaYarnTask.ClientHandler() - Constructor for class ml.shifu.guagua.yarn.GuaguaYarnTask.ClientHandler





H

HADOOP_SUCCESS - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
 
handleUpstream(ChannelHandlerContext, ChannelEvent) - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask.ClientHandler
 
HDPUtils - Class in ml.shifu.guagua.hadoop.util

For HortonWorks HDP 2.2.4 two issues not compatible with other hadoop 2.x versions.

HDPUtils() - Constructor for class ml.shifu.guagua.hadoop.util.HDPUtils
 




I

init(String[]) - Method in class ml.shifu.guagua.yarn.GuaguaYarnClient
 
initialize(GuaguaFileSplit) - Method in class ml.shifu.guagua.hadoop.io.GuaguaLineRecordReader
 
initialize(GuaguaFileSplit) - Method in class ml.shifu.guagua.hadoop.io.GuaguaSequenceAsBinaryRecordReader
 
initialize(GuaguaFileSplit) - Method in class ml.shifu.guagua.hadoop.io.GuaguaSequenceAsTextRecordReader
 
initialize(GuaguaFileSplit) - Method in class ml.shifu.guagua.hadoop.io.GuaguaSequenceRecordReader
 
initialize(GuaguaFileSplit) - Method in class ml.shifu.guagua.yarn.GuaguaLineRecordReader

Deprecated.
 
initialize(GuaguaFileSplit) - Method in class ml.shifu.guagua.yarn.GuaguaSequenceAsBinaryRecordReader

Deprecated.
 
initialize(GuaguaFileSplit) - Method in class ml.shifu.guagua.yarn.GuaguaSequenceAsTextRecordReader

Deprecated.
 
initialize(GuaguaFileSplit) - Method in class ml.shifu.guagua.yarn.GuaguaSequenceRecordReader

Deprecated.
 
InputSplitUtils - Class in ml.shifu.guagua.yarn.util

Helper class to get input splits.

isDone() - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
 
isKillContainer() - Method in class ml.shifu.guagua.yarn.GuaguaIterationStatus
 
isMaster() - Method in class ml.shifu.guagua.hadoop.io.GuaguaInputSplit
 
isMaster() - Method in class ml.shifu.guagua.yarn.GuaguaInputSplit

Deprecated.
 
isMaster() - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask
 




L

listStatus(Configuration, String) - Method in class ml.shifu.guagua.hadoop.GuaguaMRUnitDriver

List input directories.

listStatus(Configuration) - Static method in class ml.shifu.guagua.yarn.util.InputSplitUtils

List all the inputs files.





M

main(String[]) - Static method in class ml.shifu.guagua.yarn.GuaguaAppMaster

Application entry point

main(String[]) - Static method in class ml.shifu.guagua.yarn.GuaguaYarnClient
 
main(String[]) - Static method in class ml.shifu.guagua.yarn.GuaguaYarnTask
 
messageReceived(ChannelHandlerContext, MessageEvent) - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask.ClientHandler
 
ml.shifu.guagua.hadoop - package ml.shifu.guagua.hadoop

Common related Hadoop components.

ml.shifu.guagua.hadoop.io - package ml.shifu.guagua.hadoop.io
 
ml.shifu.guagua.hadoop.util - package ml.shifu.guagua.hadoop.util

Hadoop related utility functions.

ml.shifu.guagua.yarn - package ml.shifu.guagua.yarn

This package contains all the guagua-yarn implementation which is used to launch guagua master workers job on YARN 
 framework.

ml.shifu.guagua.yarn.util - package ml.shifu.guagua.yarn.util

This package contains some useful utility functions to help guagua located into YARN framework.





N

nextKeyValue() - Method in class ml.shifu.guagua.hadoop.io.GuaguaLineRecordReader
 
nextKeyValue() - Method in class ml.shifu.guagua.hadoop.io.GuaguaSequenceAsBinaryRecordReader
 
nextKeyValue() - Method in class ml.shifu.guagua.hadoop.io.GuaguaSequenceAsTextRecordReader
 
nextKeyValue() - Method in class ml.shifu.guagua.hadoop.io.GuaguaSequenceRecordReader
 
nextKeyValue() - Method in class ml.shifu.guagua.yarn.GuaguaLineRecordReader

Deprecated.
 
nextKeyValue() - Method in class ml.shifu.guagua.yarn.GuaguaSequenceAsBinaryRecordReader

Deprecated.
 
nextKeyValue() - Method in class ml.shifu.guagua.yarn.GuaguaSequenceAsTextRecordReader

Deprecated.
 
nextKeyValue() - Method in class ml.shifu.guagua.yarn.GuaguaSequenceRecordReader

Deprecated.
 




P

PIG_HEADER - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
 
PIG_SCHEMA - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
 
postApplication(MasterContext<MASTER_RESULT, WORKER_RESULT>) - Method in class ml.shifu.guagua.hadoop.ZooKeeperMasterInterceptor
 
preApplication(MasterContext<MASTER_RESULT, WORKER_RESULT>) - Method in class ml.shifu.guagua.hadoop.ZooKeeperMasterInterceptor
 
preApplication(WorkerContext<MASTER_RESULT, WORKER_RESULT>) - Method in class ml.shifu.guagua.hadoop.ZooKeeperWorkerInterceptor
 
printGenericCommandUsage(PrintStream) - Static method in class ml.shifu.guagua.hadoop.io.GuaguaOptionsParser

Print the usage message for generic command-line options supported.

printGenericCommandUsage(PrintStream) - Static method in class ml.shifu.guagua.yarn.GuaguaOptionsParser

Deprecated.
Print the usage message for generic command-line options supported.





R

readFields(DataInput) - Method in class ml.shifu.guagua.hadoop.io.GuaguaInputSplit
 
readFields(DataInput) - Method in class ml.shifu.guagua.yarn.GuaguaInputSplit

Deprecated.
 
run() - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster

Coordinates all requests for guagua's worker/master task containers, and manages application liveness heartbeat,
 completion status, teardown, etc.

run(String[]) - Method in class ml.shifu.guagua.yarn.GuaguaYarnClient

To submit an app to yarn cluster and monitor the status.

run() - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask

Run master or worker service.





S

setAllocatedCount(AtomicInteger) - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
 
setAmQueue(String) - Method in class ml.shifu.guagua.yarn.GuaguaYarnClient
 
setAmRMClient(AMRMClientAsync<AMRMClient.ContainerRequest>) - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
 
setAppAttemptId(ApplicationAttemptId) - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
 
setAppAttemptId(ApplicationAttemptId) - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask
 
setAppId(ApplicationId) - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
 
setAppId(ApplicationId) - Method in class ml.shifu.guagua.yarn.GuaguaYarnClient
 
setAppId(ApplicationId) - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask
 
setAppName(String) - Method in class ml.shifu.guagua.yarn.GuaguaYarnClient
 
setCompletedCount(AtomicInteger) - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
 
setContainerArgs(String) - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
 
setContainerId(ContainerId) - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
 
setContainerId(ContainerId) - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask
 
setContainerListener(GuaguaAppMaster.NMCallbackHandler) - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
 
setContainersToLaunch(int) - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
 
setCurrentIteration(int) - Method in class ml.shifu.guagua.yarn.GuaguaIterationStatus
 
setDone(boolean) - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
 
setExecutor(ExecutorService) - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
 
setFailedCount(AtomicInteger) - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
 
setFileSplits(FileSplit[]) - Method in class ml.shifu.guagua.hadoop.io.GuaguaInputSplit
 
setFileSplits(FileSplit[]) - Method in class ml.shifu.guagua.yarn.GuaguaInputSplit

Deprecated.
 
setGuaguaService(GuaguaService) - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask
 
setHeapPerContainer(int) - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
 
setInputSplit(GuaguaInputSplit) - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask
 
setKillContainer(boolean) - Method in class ml.shifu.guagua.yarn.GuaguaIterationStatus
 
setMaster(boolean) - Method in class ml.shifu.guagua.hadoop.io.GuaguaInputSplit
 
setMaster(boolean) - Method in class ml.shifu.guagua.yarn.GuaguaInputSplit

Deprecated.
 
setMaster(boolean) - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask
 
setNmClientAsync(NMClientAsync) - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
 
setPartition(int) - Method in class ml.shifu.guagua.yarn.GuaguaIterationStatus
 
setPartition(int) - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask
 
setStartTime(long) - Method in class ml.shifu.guagua.yarn.GuaguaYarnClient
 
setSuccessfulCount(AtomicInteger) - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
 
setTime(long) - Method in class ml.shifu.guagua.yarn.GuaguaIterationStatus
 
setTotalIteration(int) - Method in class ml.shifu.guagua.yarn.GuaguaIterationStatus
 
setup() - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask

Set up guagua service

setWritable(W) - Method in class ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter
 
setWritable(W) - Method in class ml.shifu.guagua.yarn.GuaguaWritableAdapter

Deprecated.
 
setYarnConf(YarnConfiguration) - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
 
setYarnConf(YarnConfiguration) - Method in class ml.shifu.guagua.yarn.GuaguaYarnTask
 
shutdown() - Method in class ml.shifu.guagua.yarn.GuaguaAppMaster
 
SPLIT_SLOP - Static variable in class ml.shifu.guagua.yarn.GuaguaYarnConstants
 




T

toJson(Object) - Static method in class ml.shifu.guagua.yarn.util.GsonUtils

Serialize an object to JSON-format string.

toString() - Method in class ml.shifu.guagua.hadoop.io.GuaguaInputSplit
 
toString() - Method in class ml.shifu.guagua.hadoop.io.GuaguaWritableAdapter
 
toString() - Method in class ml.shifu.guagua.yarn.GuaguaInputSplit

Deprecated.
 
toString() - Method in class ml.shifu.guagua.yarn.GuaguaIterationStatus
 
toString() - Method in class ml.shifu.guagua.yarn.GuaguaWritableAdapter

Deprecated.
 




W

write(DataOutput) - Method in class ml.shifu.guagua.hadoop.io.GuaguaInputSplit
 
write(DataOutput) - Method in class ml.shifu.guagua.yarn.GuaguaInputSplit

Deprecated.
 




Y

YarnUtils - Class in ml.shifu.guagua.yarn.util

YarnUtils is yarn-related helper class.





Z

ZooKeeperMasterInterceptor<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable> - Class in ml.shifu.guagua.hadoop

To start zookeeper instance in cluster environment.

ZooKeeperMasterInterceptor() - Constructor for class ml.shifu.guagua.hadoop.ZooKeeperMasterInterceptor
 
ZooKeeperWorkerInterceptor<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable> - Class in ml.shifu.guagua.hadoop

Follow with ZooKeeperMasterInterceptor, ZooKeeperWorkerInterceptor is used to read zookeeper server
 info from hdfs file.

ZooKeeperWorkerInterceptor() - Constructor for class ml.shifu.guagua.hadoop.ZooKeeperWorkerInterceptor
 

A B C D E F G H I L M N P R S T W Y Z