Package alluxio.multi.process
Class MultiProcessCluster
- java.lang.Object
-
- alluxio.multi.process.MultiProcessCluster
-
@ThreadSafe public final class MultiProcessCluster extends java.lang.ObjectClass for starting, stopping, and interacting with an Alluxio cluster where each master and worker runs in its own process. Compared toLocalAlluxioCluster,MultiProcessClusteris - Slower - Black box testing only (No access to master/worker internals) - Destructible (You can kill -9 masters/workers) - More realistic of real deployments Due to the slower speed, [@link LocalAlluxioCluster} should generally be preferred.MultiProcessClusteris primarily for tests which want to stop or restart servers. The synchronization strategy for this class is to synchronize all public methods.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static classMultiProcessCluster.BuilderBuilder forMultiProcessCluster.static classMultiProcessCluster.DeployModeDeploy mode for the cluster.
-
Field Summary
Fields Modifier and Type Field Description static java.lang.StringALLUXIO_USE_FIXED_TEST_PORTSstatic intPORTS_PER_MASTERstatic intPORTS_PER_WORKER
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description voidaddExternalMasterAddress(MasterNetAddress externalMasterAddress)Updates internal master list with an external address.voiddestroy()Destroys the cluster.voidformatJournal()Formats the cluster journal.ClientsgetClients()MultiProcessCluster.DeployModegetDeployMode()alluxio.client.file.FileSystemgetFileSystemClient()alluxio.client.file.FileSystemContextgetFilesystemContext()java.lang.StringgetJournalDir()java.lang.StringgetJournalDir(int masterId)alluxio.client.journal.JournalMasterClientgetJournalMasterClientForMaster()java.util.List<MasterNetAddress>getMasterAddresses()alluxio.master.MasterInquireClientgetMasterInquireClient()alluxio.client.meta.MetaMasterClientgetMetaMasterClient()alluxio.client.metrics.MetricsMasterClientgetMetricsMasterClient()intgetPrimaryMasterIndex(int timeoutMs)Gets the index of the primary master.static MultiProcessCluster.BuildernewBuilder(java.util.List<PortCoordination.ReservedPort> reservedPorts)voidnotifySuccess()Informs the cluster that the test succeeded.voidrestartZk()Restarts the Zookeeper cluster.voidsaveWorkdir()Copies the work directory to the artifacts folder.voidstart()Starts the cluster, launching all server processes.voidstartMaster(int i)Starts the specified master.voidstartMasters()Starts all masters.voidstartWorker(int i)Starts the specified worker.voidstopMaster(int i)voidstopMasters()Stops all masters.voidstopWorker(int i)voidstopZk()Stops the Zookeeper cluster.voidupdateDeployMode(MultiProcessCluster.DeployMode mode)Updates the cluster's deploy mode.voidupdateMasterConf(alluxio.conf.PropertyKey key, java.lang.String value)Updates master configuration for all masters.voidwaitForAllNodesRegistered(int timeoutMs)Waits for all nodes to be registered.intwaitForAndKillPrimaryMaster(int timeoutMs)Kills the primary master.
-
-
-
Field Detail
-
ALLUXIO_USE_FIXED_TEST_PORTS
public static final java.lang.String ALLUXIO_USE_FIXED_TEST_PORTS
- See Also:
- Constant Field Values
-
PORTS_PER_MASTER
public static final int PORTS_PER_MASTER
- See Also:
- Constant Field Values
-
PORTS_PER_WORKER
public static final int PORTS_PER_WORKER
- See Also:
- Constant Field Values
-
-
Method Detail
-
start
public void start() throws java.lang.ExceptionStarts the cluster, launching all server processes.- Throws:
java.lang.Exception
-
waitForAndKillPrimaryMaster
public int waitForAndKillPrimaryMaster(int timeoutMs) throws java.util.concurrent.TimeoutException, java.lang.InterruptedExceptionKills the primary master. If no master is currently primary, this method blocks until a primary has been elected, then kills it.- Parameters:
timeoutMs- maximum amount of time to wait, in milliseconds- Returns:
- the ID of the killed master
- Throws:
java.util.concurrent.TimeoutExceptionjava.lang.InterruptedException
-
getPrimaryMasterIndex
public int getPrimaryMasterIndex(int timeoutMs) throws java.util.concurrent.TimeoutException, java.lang.InterruptedExceptionGets the index of the primary master.- Parameters:
timeoutMs- maximum amount of time to wait, in milliseconds- Returns:
- the index of the primary master
- Throws:
java.util.concurrent.TimeoutExceptionjava.lang.InterruptedException
-
waitForAllNodesRegistered
public void waitForAllNodesRegistered(int timeoutMs) throws java.util.concurrent.TimeoutException, java.lang.InterruptedExceptionWaits for all nodes to be registered.- Parameters:
timeoutMs- maximum amount of time to wait, in milliseconds- Throws:
java.util.concurrent.TimeoutExceptionjava.lang.InterruptedException
-
getDeployMode
public MultiProcessCluster.DeployMode getDeployMode()
- Returns:
- the deploy mode
-
getFilesystemContext
public alluxio.client.file.FileSystemContext getFilesystemContext()
- Returns:
- the
FileSystemContextwhich can be used to access the cluster
-
getFileSystemClient
public alluxio.client.file.FileSystem getFileSystemClient()
- Returns:
- a client for interacting with the cluster
-
getMetaMasterClient
public alluxio.client.meta.MetaMasterClient getMetaMasterClient()
- Returns:
- a meta master client
-
getMetricsMasterClient
public alluxio.client.metrics.MetricsMasterClient getMetricsMasterClient()
- Returns:
- a metrics master client
-
getJournalMasterClientForMaster
public alluxio.client.journal.JournalMasterClient getJournalMasterClientForMaster()
- Returns:
- a journal master client for alluxio master
-
getClients
public Clients getClients()
- Returns:
- clients for communicating with the cluster
-
notifySuccess
public void notifySuccess()
Informs the cluster that the test succeeded. If this method is never called, the cluster will save a copy of its state during teardown.
-
saveWorkdir
public void saveWorkdir() throws java.io.IOExceptionCopies the work directory to the artifacts folder.- Throws:
java.io.IOException
-
destroy
public void destroy() throws java.io.IOExceptionDestroys the cluster. It may not be re-started after being destroyed.- Throws:
java.io.IOException
-
startMasters
public void startMasters()
Starts all masters.
-
startMaster
public void startMaster(int i) throws java.io.IOExceptionStarts the specified master.- Parameters:
i- the index of the master to start- Throws:
java.io.IOException
-
startWorker
public void startWorker(int i) throws java.io.IOExceptionStarts the specified worker.- Parameters:
i- the index of the worker to start- Throws:
java.io.IOException
-
stopMasters
public void stopMasters()
Stops all masters.
-
stopMaster
public void stopMaster(int i) throws java.io.IOException- Parameters:
i- the index of the master to stop- Throws:
java.io.IOException
-
updateMasterConf
public void updateMasterConf(alluxio.conf.PropertyKey key, @Nullable java.lang.String value)Updates master configuration for all masters. This will take effect on a master the next time the master is started.- Parameters:
key- the key to updatevalue- the value to set, or null to unset the key
-
addExternalMasterAddress
public void addExternalMasterAddress(MasterNetAddress externalMasterAddress)
Updates internal master list with an external address. This API is to provide support when this cluster is joined by external masters. Calling this API with an external master address will make this cluster aware of it when creating clients/contexts etc.- Parameters:
externalMasterAddress- external master address
-
updateDeployMode
public void updateDeployMode(MultiProcessCluster.DeployMode mode)
Updates the cluster's deploy mode.- Parameters:
mode- the mode to set
-
stopWorker
public void stopWorker(int i) throws java.io.IOException- Parameters:
i- the index of the worker to stop- Throws:
java.io.IOException
-
getJournalDir
public java.lang.String getJournalDir()
- Returns:
- the journal directory
-
getJournalDir
public java.lang.String getJournalDir(int masterId)
- Parameters:
masterId- index of the master- Returns:
- the journal directory for the specified master
-
getMasterAddresses
public java.util.List<MasterNetAddress> getMasterAddresses()
- Returns:
- return the list of master addresses
-
stopZk
public void stopZk() throws java.io.IOExceptionStops the Zookeeper cluster.- Throws:
java.io.IOException
-
restartZk
public void restartZk() throws java.lang.ExceptionRestarts the Zookeeper cluster.- Throws:
java.lang.Exception
-
formatJournal
public void formatJournal() throws java.io.IOExceptionFormats the cluster journal.- Throws:
java.io.IOException
-
getMasterInquireClient
public alluxio.master.MasterInquireClient getMasterInquireClient()
- Returns:
- a client for determining the serving master
-
newBuilder
public static MultiProcessCluster.Builder newBuilder(java.util.List<PortCoordination.ReservedPort> reservedPorts)
- Parameters:
reservedPorts- ports reserved for usage by this cluster- Returns:
- a new builder for an
MultiProcessCluster
-
-