ml.shifu.guagua.yarn

Class GuaguaYarnTask<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>

java.lang.Object

ml.shifu.guagua.yarn.GuaguaYarnTask<MASTER_RESULT,WORKER_RESULT>

public class GuaguaYarnTask<MASTER_RESULT extends Bytable,WORKER_RESULT extends Bytable>
extends Object

GuaguaYarnTask is a entry point to run both master and workers.

partition should be passed as the last parameter in main. And it should be not be changed if we try another task.

Input split are now storing in guagua-conf.xml. We read data from there and check whether this task is master or worker.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	GuaguaYarnTask.ClientHandler ClientHandeler used to update progress to RPC server (AppMaster).

Constructor Summary

Constructors

Constructor and Description
GuaguaYarnTask(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId, org.apache.hadoop.yarn.api.records.ContainerId containerId, int partition, String rpcHostName, String rpcPort, org.apache.hadoop.conf.Configuration conf) Constructor with yarn task related parameters.

Method Summary

Methods

Modifier and Type	Method and Description
protected void	cleanup() Clean up resources used
org.apache.hadoop.yarn.api.records.ApplicationAttemptId	getAppAttemptId()
org.apache.hadoop.yarn.api.records.ApplicationId	getAppId()
org.apache.hadoop.yarn.api.records.ContainerId	getContainerId()
GuaguaService	getGuaguaService()
GuaguaInputSplit	getInputSplit()
int	getPartition()
org.apache.hadoop.conf.Configuration	getYarnConf()
boolean	isMaster()
static void	main(String[] args)
void	run() Run master or worker service.
void	setAppAttemptId(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId)
void	setAppId(org.apache.hadoop.yarn.api.records.ApplicationId appId)
void	setContainerId(org.apache.hadoop.yarn.api.records.ContainerId containerId)
void	setGuaguaService(GuaguaService guaguaService)
void	setInputSplit(GuaguaInputSplit inputSplit)
void	setMaster(boolean isMaster)
void	setPartition(int partition)
protected void	setup() Set up guagua service
void	setYarnConf(org.apache.hadoop.yarn.conf.YarnConfiguration yarnConf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GuaguaYarnTask

public GuaguaYarnTask(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId,
              org.apache.hadoop.yarn.api.records.ContainerId containerId,
              int partition,
              String rpcHostName,
              String rpcPort,
              org.apache.hadoop.conf.Configuration conf)

Constructor with yarn task related parameters.

Method Detail

setup

protected void setup()

Set up guagua service

run

public void run()

Run master or worker service.

cleanup

protected void cleanup()

Clean up resources used

getGuaguaService

public GuaguaService getGuaguaService()

setGuaguaService

public void setGuaguaService(GuaguaService guaguaService)

getPartition

public int getPartition()

setPartition

public void setPartition(int partition)

getAppAttemptId

public org.apache.hadoop.yarn.api.records.ApplicationAttemptId getAppAttemptId()

setAppAttemptId

public void setAppAttemptId(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId)

getContainerId

public org.apache.hadoop.yarn.api.records.ContainerId getContainerId()

setContainerId

public void setContainerId(org.apache.hadoop.yarn.api.records.ContainerId containerId)

isMaster

public boolean isMaster()

setMaster

public void setMaster(boolean isMaster)

getYarnConf

public org.apache.hadoop.conf.Configuration getYarnConf()

setYarnConf

public void setYarnConf(org.apache.hadoop.yarn.conf.YarnConfiguration yarnConf)

getAppId

public org.apache.hadoop.yarn.api.records.ApplicationId getAppId()

setAppId

public void setAppId(org.apache.hadoop.yarn.api.records.ApplicationId appId)

getInputSplit

public GuaguaInputSplit getInputSplit()

setInputSplit

public void setInputSplit(GuaguaInputSplit inputSplit)

main

public static void main(String[] args)