ml.shifu.guagua.yarn.util
类 YarnUtils

java.lang.Object
  ml.shifu.guagua.yarn.util.YarnUtils

public final class YarnUtils
extends Object

YarnUtils is yarn-related helper class.

字段摘要

static String

GUAGUA_YARN_TMP

方法摘要

static void	addFileToResourceMap(Map<String,org.apache.hadoop.yarn.api.records.LocalResource> localResources, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path target) Boilerplate to add a file to the local resources..
static void	addLocalClasspathToEnv(Map<String,String> env, org.apache.hadoop.conf.Configuration conf) Populate the environment string map to be added to the environment vars in a remote execution container.
static void	copyLocalResourcesToFs(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.yarn.api.records.ApplicationId appId) Copy batch of resources to hdfs app directory for master and containers.
static void	copyLocalResourceToFs(String src, String dst, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.yarn.api.records.ApplicationId appId) Copy local file to a named-file in remote FS
static void	exportGuaguaConfiguration(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.yarn.api.records.ApplicationId appId) Export our populated Configuration as an XML file to be used by the ApplicationMaster's exec container, and register it with LocalResources.
static org.apache.hadoop.fs.Path	getAppDirectory(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.yarn.api.records.ApplicationId appId) Working folder to store jars, files and other resources
static List<String>	getCommand(String mainClass, String vmArgs, String programArgs, String memory) Return java command according to main class, vm args, program args and memory settings.
static Map<String,org.apache.hadoop.yarn.api.records.LocalResource>	getLocalResourceMap(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.yarn.api.records.ApplicationId appId) Build local resources including main app jar, lib jars, log4j.properties and guagua-conf.xml.
static org.apache.hadoop.fs.Path	getPathForResource(org.apache.hadoop.fs.FileSystem fs, String loc, org.apache.hadoop.yarn.api.records.ApplicationId appId) Get path to store local resources on hdfs.

从类 java.lang.Object 继承的方法

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细信息

GUAGUA_YARN_TMP

public static final String GUAGUA_YARN_TMP

另请参见：

常量字段值

方法详细信息

getLocalResourceMap

public static Map<String,org.apache.hadoop.yarn.api.records.LocalResource> getLocalResourceMap(org.apache.hadoop.conf.Configuration conf,
                                                                                               org.apache.hadoop.yarn.api.records.ApplicationId appId)
                                                                                        throws IOException

Build local resources including main app jar, lib jars, log4j.properties and guagua-conf.xml.

抛出：

IOException

getCommand

public static List<String> getCommand(String mainClass,
                                      String vmArgs,
                                      String programArgs,
                                      String memory)

Return java command according to main class, vm args, program args and memory settings.

addLocalClasspathToEnv

public static void addLocalClasspathToEnv(Map<String,String> env,
                                          org.apache.hadoop.conf.Configuration conf)

Populate the environment string map to be added to the environment vars in a remote execution container.

参数：

env - the map of env var values.

conf - the Configuration to pull values from.

getPathForResource

public static org.apache.hadoop.fs.Path getPathForResource(org.apache.hadoop.fs.FileSystem fs,
                                                           String loc,
                                                           org.apache.hadoop.yarn.api.records.ApplicationId appId)

Get path to store local resources on hdfs.

getAppDirectory

public static org.apache.hadoop.fs.Path getAppDirectory(org.apache.hadoop.fs.FileSystem fs,
                                                        org.apache.hadoop.yarn.api.records.ApplicationId appId)

Working folder to store jars, files and other resources

copyLocalResourcesToFs

public static void copyLocalResourcesToFs(org.apache.hadoop.conf.Configuration conf,
                                          org.apache.hadoop.yarn.api.records.ApplicationId appId)
                                   throws IOException

Copy batch of resources to hdfs app directory for master and containers.

抛出：

IOException

addFileToResourceMap

public static void addFileToResourceMap(Map<String,org.apache.hadoop.yarn.api.records.LocalResource> localResources,
                                        org.apache.hadoop.fs.FileSystem fs,
                                        org.apache.hadoop.fs.Path target)
                                 throws IOException

Boilerplate to add a file to the local resources..

参数：

localResources - the LocalResources map to populate.

fs - handle to the HDFS file system.

target - the file to send to the remote container.

抛出：

IOException

copyLocalResourceToFs

public static void copyLocalResourceToFs(String src,
                                         String dst,
                                         org.apache.hadoop.conf.Configuration conf,
                                         org.apache.hadoop.yarn.api.records.ApplicationId appId)
                                  throws IOException

Copy local file to a named-file in remote FS

抛出：

IOException

exportGuaguaConfiguration

public static void exportGuaguaConfiguration(org.apache.hadoop.conf.Configuration conf,
                                             org.apache.hadoop.yarn.api.records.ApplicationId appId)
                                      throws IOException

Export our populated Configuration as an XML file to be used by the ApplicationMaster's exec container, and register it with LocalResources.

参数：

conf - the current Configuration object to be published.

appId - the ApplicationId to stamp this app's base HDFS resources dir.

抛出：

IOException