org.apache.hadoop.hbase.master.assignment

Class RegionTransitionProcedure

java.lang.Object

org.apache.hadoop.hbase.procedure2.Procedure<MasterProcedureEnv>

org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure

All Implemented Interfaces:

Comparable<org.apache.hadoop.hbase.procedure2.Procedure<MasterProcedureEnv>>, TableProcedureInterface, org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.ServerName>

Direct Known Subclasses:

AssignProcedure, UnassignProcedure

@InterfaceAudience.Private
public abstract class RegionTransitionProcedure
extends org.apache.hadoop.hbase.procedure2.Procedure<MasterProcedureEnv>
implements TableProcedureInterface, org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.ServerName>

Base class for the Assign and Unassign Procedure. Locking: Takes exclusive lock on the region being assigned/unassigned. Thus, there can only be one RegionTransitionProcedure per region running at a time (see MasterProcedureScheduler).

This procedure is asynchronous and responds to external events. The AssignmentManager will notify this procedure when the RS completes the operation and reports the transitioned state (see the Assign and Unassign class for more detail).

Procedures move from the REGION_TRANSITION_QUEUE state when they are first submitted, to the REGION_TRANSITION_DISPATCH state when the request to remote server is sent and the Procedure is suspended waiting on external event to be woken again. Once the external event is triggered, Procedure moves to the REGION_TRANSITION_FINISH state.

NOTE: AssignProcedure and UnassignProcedure should not be thought of as being asymmetric, at least currently.

• AssignProcedure moves through all the above described states and implements methods associated with each while UnassignProcedure starts at state REGION_TRANSITION_DISPATCH and state REGION_TRANSITION_QUEUE is not supported.
• When any step in AssignProcedure fails, failure handler AssignProcedure#handleFailure(MasterProcedureEnv, RegionStateNode) re-attempts the assignment by setting the procedure state to REGION_TRANSITION_QUEUE and forces assignment to a different target server by setting AssignProcedure.forceNewPlan. When the number of attempts reaches threshold configuration 'hbase.assignment.maximum.attempts', the procedure is aborted. For UnassignProcedure, similar re-attempts are intentionally not implemented. It is a 'one shot' procedure. See its class doc for how it handles failure.
• If we find a region in an 'unexpected' state, we'll complain and retry with backoff forever. The 'unexpected' state needs to be fixed either by another running Procedure or by operator intervention (Regions in 'unexpected' state indicates bug or unexpected transition type). For this to work, subclasses need to persist the 'attempt' counter kept in this class when they do serializeStateData and restore it inside their deserializeStateData, just as they do for regionInfo.

TODO: Considering it is a priority doing all we can to get make a region available as soon as possible, re-attempting with any target makes sense if specified target fails in case of AssignProcedure. For UnassignProcedure, our concern is preventing data loss on failed unassign. See class doc for explanation.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.procedure2.Procedure

org.apache.hadoop.hbase.procedure2.Procedure.LockState

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.master.procedure.TableProcedureInterface

TableProcedureInterface.TableOperationType

Field Summary

Fields

Modifier and Type	Field and Description
protected AtomicBoolean	aborted

Fields inherited from class org.apache.hadoop.hbase.procedure2.Procedure

NO_PROC_ID, NO_TIMEOUT

Constructor Summary

Constructors

Constructor and Description
RegionTransitionProcedure()
RegionTransitionProcedure(org.apache.hadoop.hbase.client.RegionInfo regionInfo, boolean override)

Method Summary

Modifier and Type	Method and Description
protected boolean	abort(MasterProcedureEnv env)
protected org.apache.hadoop.hbase.procedure2.Procedure.LockState	acquireLock(MasterProcedureEnv env)
protected boolean	addToRemoteDispatcher(MasterProcedureEnv env, org.apache.hadoop.hbase.ServerName targetServer) Be careful! At the end of this method, the procedure has either succeeded and this procedure has been set into a suspended state OR, we failed and this procedure has been put back on the scheduler ready for another worker to pick it up.
protected void	bypass(MasterProcedureEnv env)
protected org.apache.hadoop.hbase.procedure2.Procedure[]	execute(MasterProcedureEnv env)
protected abstract void	finishTransition(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode)
protected int	getAttempt()
org.apache.hadoop.hbase.client.RegionInfo	getRegionInfo()
RegionStates.RegionStateNode	getRegionState(MasterProcedureEnv env)
abstract org.apache.hadoop.hbase.ServerName	getServer(MasterProcedureEnv env) Used by ServerCrashProcedure to see if this Assign/Unassign needs processing.
org.apache.hadoop.hbase.TableName	getTableName()
protected org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RegionTransitionState	getTransitionState()
protected boolean	holdLock(MasterProcedureEnv env)
boolean	isMeta()
protected abstract boolean	isRollbackSupported(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RegionTransitionState state)
protected boolean	isServerOnline(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode)
protected boolean	isServerOnline(MasterProcedureEnv env, org.apache.hadoop.hbase.ServerName serverName)
protected void	releaseLock(MasterProcedureEnv env)
abstract org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher.RemoteOperation	remoteCallBuild(MasterProcedureEnv env, org.apache.hadoop.hbase.ServerName serverName)
protected abstract boolean	remoteCallFailed(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode, IOException exception)
boolean	remoteCallFailed(MasterProcedureEnv env, org.apache.hadoop.hbase.ServerName serverName, IOException exception)
void	remoteOperationCompleted(MasterProcedureEnv env)
void	remoteOperationFailed(MasterProcedureEnv env, org.apache.hadoop.hbase.procedure2.RemoteProcedureException error)
protected abstract void	reportTransition(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode, org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode code, long seqId)
protected void	reportTransition(MasterProcedureEnv env, org.apache.hadoop.hbase.ServerName serverName, org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode code, long seqId)
protected void	rollback(MasterProcedureEnv env)
protected void	setAttempt(int attempt) This setter is for subclasses to call in their Procedure.deserializeStateData(ProcedureStateSerializer) method.
protected void	setOverride(boolean override) This setter is for subclasses to call in their Procedure.deserializeStateData(ProcedureStateSerializer) method.
protected void	setRegionInfo(org.apache.hadoop.hbase.client.RegionInfo regionInfo) This setter is for subclasses to call in their Procedure.deserializeStateData(ProcedureStateSerializer) method.
protected boolean	setTimeoutFailure(MasterProcedureEnv env) At end of timeout, wake ourselves up so we run again.
protected boolean	shouldWaitClientAck(MasterProcedureEnv env)
protected abstract boolean	startTransition(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode)
void	toStringClassDetails(StringBuilder sb)
protected void	toStringState(StringBuilder builder)
protected abstract boolean	updateTransition(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode) Called when the Procedure is in the REGION_TRANSITION_DISPATCH state.
protected boolean	waitInitialized(MasterProcedureEnv env)

Methods inherited from class org.apache.hadoop.hbase.procedure2.Procedure

addStackIndex, afterReplay, beforeReplay, compareTo, completionCleanup, deserializeStateData, doExecute, doRollback, elapsedTime, getChildrenLatch, getException, getLastUpdate, getNonceKey, getOwner, getParentProcId, getProcedureMetrics, getProcId, getProcIdHashCode, getProcName, getResult, getRootProcedureId, getRootProcId, getStackIndexes, getState, getSubmittedTime, getTimeout, getTimeoutTimestamp, hasChildren, hasException, hasLock, hasOwner, hasParent, hasTimeout, haveSameParent, incChildrenLatch, isBypass, isFailed, isFinished, isInitializing, isLockedWhenLoading, isRunnable, isSuccess, isWaiting, isYieldAfterExecutionStep, removeStackIndex, serializeStateData, setAbortFailure, setChildrenLatch, setFailure, setFailure, setLastUpdate, setNonceKey, setOwner, setOwner, setParentProcId, setProcId, setResult, setRootProcId, setStackIndexes, setState, setSubmittedTime, setTimeout, skipPersistence, toString, toStringClass, toStringDetails, toStringSimpleSB, updateMetricsOnFinish, updateMetricsOnSubmit, updateTimestamp, wasExecuted

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.master.procedure.TableProcedureInterface

getTableOperationType

Field Detail

aborted

protected final AtomicBoolean aborted

Constructor Detail

RegionTransitionProcedure

public RegionTransitionProcedure()

RegionTransitionProcedure

public RegionTransitionProcedure(org.apache.hadoop.hbase.client.RegionInfo regionInfo,
                                 boolean override)

Method Detail

getRegionInfo

public org.apache.hadoop.hbase.client.RegionInfo getRegionInfo()

setRegionInfo

protected void setRegionInfo(org.apache.hadoop.hbase.client.RegionInfo regionInfo)

This setter is for subclasses to call in their Procedure.deserializeStateData(ProcedureStateSerializer) method. Expectation is that subclasses will persist `regioninfo` in their Procedure.serializeStateData(ProcedureStateSerializer) method and then restore `regionInfo` on deserialization by calling this.

setOverride

protected void setOverride(boolean override)

This setter is for subclasses to call in their Procedure.deserializeStateData(ProcedureStateSerializer) method. Expectation is that subclasses will persist `override` in their Procedure.serializeStateData(ProcedureStateSerializer) method and then restore `override` on deserialization by calling this.

setAttempt

protected void setAttempt(int attempt)

This setter is for subclasses to call in their Procedure.deserializeStateData(ProcedureStateSerializer) method.

See Also:

setRegionInfo(RegionInfo)

getAttempt

protected int getAttempt()

getTableName

public org.apache.hadoop.hbase.TableName getTableName()

Specified by:

getTableName in interface TableProcedureInterface

Returns:

the name of the table the procedure is operating on

isMeta

public boolean isMeta()

toStringClassDetails

public void toStringClassDetails(StringBuilder sb)

Overrides:

toStringClassDetails in class org.apache.hadoop.hbase.procedure2.Procedure<MasterProcedureEnv>

getRegionState

public RegionStates.RegionStateNode getRegionState(MasterProcedureEnv env)

getTransitionState

protected org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RegionTransitionState getTransitionState()

startTransition

protected abstract boolean startTransition(MasterProcedureEnv env,
                                           RegionStates.RegionStateNode regionNode)
                                    throws IOException,
                                           org.apache.hadoop.hbase.procedure2.ProcedureSuspendedException

Throws:

IOException

org.apache.hadoop.hbase.procedure2.ProcedureSuspendedException

updateTransition

protected abstract boolean updateTransition(MasterProcedureEnv env,
                                            RegionStates.RegionStateNode regionNode)
                                     throws IOException,
                                            org.apache.hadoop.hbase.procedure2.ProcedureSuspendedException

Called when the Procedure is in the REGION_TRANSITION_DISPATCH state. In here we do the RPC call to OPEN/CLOSE the region. The suspending of the thread so it sleeps until it gets update that the OPEN/CLOSE has succeeded is complicated. Read the implementations to learn more.

Throws:

IOException

org.apache.hadoop.hbase.procedure2.ProcedureSuspendedException

finishTransition

protected abstract void finishTransition(MasterProcedureEnv env,
                                         RegionStates.RegionStateNode regionNode)
                                  throws IOException,
                                         org.apache.hadoop.hbase.procedure2.ProcedureSuspendedException

Throws:

IOException

org.apache.hadoop.hbase.procedure2.ProcedureSuspendedException

reportTransition

protected abstract void reportTransition(MasterProcedureEnv env,
                                         RegionStates.RegionStateNode regionNode,
                                         org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode code,
                                         long seqId)
                                  throws org.apache.hadoop.hbase.exceptions.UnexpectedStateException

Throws:

org.apache.hadoop.hbase.exceptions.UnexpectedStateException

remoteCallBuild

public abstract org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher.RemoteOperation remoteCallBuild(MasterProcedureEnv env,
                                                                                                             org.apache.hadoop.hbase.ServerName serverName)

Specified by:

remoteCallBuild in interface org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.ServerName>

remoteCallFailed

protected abstract boolean remoteCallFailed(MasterProcedureEnv env,
                                            RegionStates.RegionStateNode regionNode,
                                            IOException exception)

Returns:

True if processing of fail is complete; the procedure will be woken from its suspend and we'll go back to running through procedure steps: otherwise if false we leave the procedure in suspended state.

remoteCallFailed

public boolean remoteCallFailed(MasterProcedureEnv env,
                                org.apache.hadoop.hbase.ServerName serverName,
                                IOException exception)

Specified by:

remoteCallFailed in interface org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.ServerName>

addToRemoteDispatcher

protected boolean addToRemoteDispatcher(MasterProcedureEnv env,
                                        org.apache.hadoop.hbase.ServerName targetServer)

Be careful! At the end of this method, the procedure has either succeeded and this procedure has been set into a suspended state OR, we failed and this procedure has been put back on the scheduler ready for another worker to pick it up. In both cases, we need to exit the current Worker processing immediately!

Returns:

True if we successfully dispatched the call and false if we failed; if failed, we need to roll back any setup done for the dispatch.

reportTransition

protected void reportTransition(MasterProcedureEnv env,
                                org.apache.hadoop.hbase.ServerName serverName,
                                org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionStateTransition.TransitionCode code,
                                long seqId)
                         throws org.apache.hadoop.hbase.exceptions.UnexpectedStateException

Throws:

org.apache.hadoop.hbase.exceptions.UnexpectedStateException

isServerOnline

protected boolean isServerOnline(MasterProcedureEnv env,
                                 RegionStates.RegionStateNode regionNode)

isServerOnline

protected boolean isServerOnline(MasterProcedureEnv env,
                                 org.apache.hadoop.hbase.ServerName serverName)

toStringState

protected void toStringState(StringBuilder builder)

Overrides:

toStringState in class org.apache.hadoop.hbase.procedure2.Procedure<MasterProcedureEnv>

execute

protected org.apache.hadoop.hbase.procedure2.Procedure[] execute(MasterProcedureEnv env)
                                                          throws org.apache.hadoop.hbase.procedure2.ProcedureSuspendedException

Specified by:

execute in class org.apache.hadoop.hbase.procedure2.Procedure<MasterProcedureEnv>

Throws:

org.apache.hadoop.hbase.procedure2.ProcedureSuspendedException

setTimeoutFailure

protected boolean setTimeoutFailure(MasterProcedureEnv env)

At end of timeout, wake ourselves up so we run again.

Overrides:

setTimeoutFailure in class org.apache.hadoop.hbase.procedure2.Procedure<MasterProcedureEnv>

rollback

protected void rollback(MasterProcedureEnv env)

Specified by:

rollback in class org.apache.hadoop.hbase.procedure2.Procedure<MasterProcedureEnv>

isRollbackSupported

protected abstract boolean isRollbackSupported(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.RegionTransitionState state)

abort

protected boolean abort(MasterProcedureEnv env)

Specified by:

abort in class org.apache.hadoop.hbase.procedure2.Procedure<MasterProcedureEnv>

waitInitialized

protected boolean waitInitialized(MasterProcedureEnv env)

Overrides:

waitInitialized in class org.apache.hadoop.hbase.procedure2.Procedure<MasterProcedureEnv>

acquireLock

protected org.apache.hadoop.hbase.procedure2.Procedure.LockState acquireLock(MasterProcedureEnv env)

Overrides:

acquireLock in class org.apache.hadoop.hbase.procedure2.Procedure<MasterProcedureEnv>

releaseLock

protected void releaseLock(MasterProcedureEnv env)

Overrides:

releaseLock in class org.apache.hadoop.hbase.procedure2.Procedure<MasterProcedureEnv>

holdLock

protected boolean holdLock(MasterProcedureEnv env)

Overrides:

holdLock in class org.apache.hadoop.hbase.procedure2.Procedure<MasterProcedureEnv>

shouldWaitClientAck

protected boolean shouldWaitClientAck(MasterProcedureEnv env)

Overrides:

shouldWaitClientAck in class org.apache.hadoop.hbase.procedure2.Procedure<MasterProcedureEnv>

getServer

public abstract org.apache.hadoop.hbase.ServerName getServer(MasterProcedureEnv env)

Used by ServerCrashProcedure to see if this Assign/Unassign needs processing.

Returns:

ServerName the Assign or Unassign is going against.

remoteOperationCompleted

public void remoteOperationCompleted(MasterProcedureEnv env)

Specified by:

remoteOperationCompleted in interface org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.ServerName>

remoteOperationFailed

public void remoteOperationFailed(MasterProcedureEnv env,
                                  org.apache.hadoop.hbase.procedure2.RemoteProcedureException error)

Specified by:

remoteOperationFailed in interface org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.ServerName>

bypass

protected void bypass(MasterProcedureEnv env)

Overrides:

bypass in class org.apache.hadoop.hbase.procedure2.Procedure<MasterProcedureEnv>