org.apache.hadoop.hbase.procedure2.store.wal

Class WALProcedureStore

java.lang.Object

org.apache.hadoop.hbase.procedure2.store.ProcedureStoreBase

org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore

All Implemented Interfaces:

ProcedureStore

@InterfaceAudience.Private
public class WALProcedureStore
extends ProcedureStoreBase

WAL implementation of the ProcedureStore.

When starting, the upper layer will first call start(int), then recoverLease(), then ProcedureStore.load(ProcedureLoader).

In recoverLease(), we will get the lease by closing all the existing wal files(by calling recoverFileLease), and creating a new wal writer. And we will also get the list of all the old wal files.

FIXME: notice that the current recover lease implementation is problematic, it can not deal with the races if there are two master both wants to acquire the lease...

In ProcedureStore.load(ProcedureLoader) method, we will load all the active procedures. See the comments of this method for more details.

The actual logging way is a bit like our FileSystem based WAL implementation as RS side. There is a slots, which is more like the ring buffer, and in the insert, update and delete methods we will put thing into the slots and wait. And there is a background sync thread(see the syncLoop() method) which get data from the slots and write them to the FileSystem, and notify the caller that we have finished.

TODO: try using disruptor to increase performance and simplify the logic?

The storeTracker keeps track of the modified procedures in the newest wal file, which is also the one being written currently. And the deleted bits in it are for all the procedures, not only the ones in the newest wal file. And when rolling a log, we will first store it in the trailer of the current wal file, and then reset its modified bits, so that it can start to track the modified procedures for the new wal file.

The holdingCleanupTracker is used to test whether we are safe to delete the oldest wal file. When there are log rolling and there are more than 1 wal files, we will make use of it. It will first be initialized to the oldest file's tracker(which is stored in the trailer), using the method ProcedureStoreTracker.resetTo(ProcedureStoreTracker, boolean), and then merge it with the tracker of every newer wal files, using the ProcedureStoreTracker.setDeletedIfModifiedInBoth(ProcedureStoreTracker). If we find out that all the modified procedures for the oldest wal file are modified or deleted in newer wal files, then we can delete it. This is because that, every time we call ProcedureStore.insert(Procedure[]) or ProcedureStore.update(Procedure), we will persist the full state of a Procedure, so the earlier wal records for this procedure can all be deleted.

See Also:

for printing content of a single WAL., to parse a directory of MasterWALProcs.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	WALProcedureStore.LeaseRecovery
static class	WALProcedureStore.SyncMetrics

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.procedure2.store.ProcedureStore

ProcedureStore.ProcedureIterator, ProcedureStore.ProcedureLoader, ProcedureStore.ProcedureStoreListener

Field Summary

Fields

Modifier and Type	Field and Description
static String	EXEC_WAL_CLEANUP_ON_LOAD_CONF_KEY
static String	LOG_PREFIX
static String	MASTER_PROCEDURE_LOGDIR Used to construct the name of the log directory for master procedures
static String	MAX_RETRIES_BEFORE_ROLL_CONF_KEY
static String	MAX_SYNC_FAILURE_ROLL_CONF_KEY
static String	PERIODIC_ROLL_CONF_KEY
static String	ROLL_RETRIES_CONF_KEY
static String	ROLL_THRESHOLD_CONF_KEY
static String	STORE_WAL_SYNC_STATS_COUNT
static String	SYNC_WAIT_MSEC_CONF_KEY
static String	USE_HSYNC_CONF_KEY
static String	WAIT_BEFORE_ROLL_CONF_KEY
static String	WAL_COUNT_WARN_THRESHOLD_CONF_KEY

Constructor Summary

Constructors

Constructor and Description
WALProcedureStore(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path walDir, org.apache.hadoop.fs.Path walArchiveDir, WALProcedureStore.LeaseRecovery leaseRecovery)
WALProcedureStore(org.apache.hadoop.conf.Configuration conf, WALProcedureStore.LeaseRecovery leaseRecovery)

Method Summary

Modifier and Type	Method and Description
void	delete(long procId) The specified procId was removed from the executor, due to completion, abort or failure.
void	delete(long[] procIds, int offset, int count) The specified procIds were removed from the executor, due to completion, abort or failure.
void	delete(Procedure<?> proc, long[] subProcIds) The parent procedure completed.
ArrayList<ProcedureWALFile>	getActiveLogs()
Set<ProcedureWALFile>	getCorruptedLogs()
org.apache.hadoop.fs.FileSystem	getFileSystem()
protected org.apache.hadoop.fs.Path	getLogFilePath(long logId)
long	getMillisFromLastRoll()
long	getMillisToNextPeriodicRoll()
int	getNumThreads()
ProcedureStoreTracker	getStoreTracker()
ArrayList<WALProcedureStore.SyncMetrics>	getSyncMetrics()
org.apache.hadoop.fs.Path	getWALDir()
void	insert(Procedure<?>[] procs) Serialize a set of new procedures.
void	insert(Procedure<?> proc, Procedure<?>[] subprocs) When a procedure is submitted to the executor insert(proc, null) will be called.
void	load(ProcedureStore.ProcedureLoader loader) Load the Procedures in the store.
static void	main(String[] args) Parses a directory of WALs building up ProcedureState.
void	recoverLease() Acquire the lease for the procedure store.
boolean	rollWriterForTesting()
int	setRunningProcedureCount(int count) Set the number of procedure running.
void	start(int numSlots) Start/Open the procedure store
void	stop(boolean abort) Stop/Close the procedure store
protected long	syncSlots(org.apache.hadoop.fs.FSDataOutputStream stream, ByteSlot[] slots, int offset, int count)
protected void	syncStream(org.apache.hadoop.fs.FSDataOutputStream stream)
void	update(Procedure<?> proc) The specified procedure was executed, and the new state should be written to the store.

Methods inherited from class org.apache.hadoop.hbase.procedure2.store.ProcedureStoreBase

isRunning, registerListener, sendAbortProcessSignal, sendForceUpdateSignal, sendPostSyncSignal, setRunning, unregisterListener

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG_PREFIX

public static final String LOG_PREFIX

See Also:

Constant Field Values

MASTER_PROCEDURE_LOGDIR

public static final String MASTER_PROCEDURE_LOGDIR

Used to construct the name of the log directory for master procedures

See Also:

Constant Field Values

WAL_COUNT_WARN_THRESHOLD_CONF_KEY

public static final String WAL_COUNT_WARN_THRESHOLD_CONF_KEY

See Also:

Constant Field Values

EXEC_WAL_CLEANUP_ON_LOAD_CONF_KEY

public static final String EXEC_WAL_CLEANUP_ON_LOAD_CONF_KEY

See Also:

Constant Field Values

MAX_RETRIES_BEFORE_ROLL_CONF_KEY

public static final String MAX_RETRIES_BEFORE_ROLL_CONF_KEY

See Also:

Constant Field Values

WAIT_BEFORE_ROLL_CONF_KEY

public static final String WAIT_BEFORE_ROLL_CONF_KEY

See Also:

Constant Field Values

ROLL_RETRIES_CONF_KEY

public static final String ROLL_RETRIES_CONF_KEY

See Also:

Constant Field Values

MAX_SYNC_FAILURE_ROLL_CONF_KEY

public static final String MAX_SYNC_FAILURE_ROLL_CONF_KEY

See Also:

Constant Field Values

PERIODIC_ROLL_CONF_KEY

public static final String PERIODIC_ROLL_CONF_KEY

See Also:

Constant Field Values

SYNC_WAIT_MSEC_CONF_KEY

public static final String SYNC_WAIT_MSEC_CONF_KEY

See Also:

Constant Field Values

USE_HSYNC_CONF_KEY

public static final String USE_HSYNC_CONF_KEY

See Also:

Constant Field Values

ROLL_THRESHOLD_CONF_KEY

public static final String ROLL_THRESHOLD_CONF_KEY

See Also:

Constant Field Values

STORE_WAL_SYNC_STATS_COUNT

public static final String STORE_WAL_SYNC_STATS_COUNT

See Also:

Constant Field Values

Constructor Detail

WALProcedureStore

public WALProcedureStore(org.apache.hadoop.conf.Configuration conf,
                         WALProcedureStore.LeaseRecovery leaseRecovery)
                  throws IOException

Throws:

IOException

WALProcedureStore

public WALProcedureStore(org.apache.hadoop.conf.Configuration conf,
                         org.apache.hadoop.fs.Path walDir,
                         org.apache.hadoop.fs.Path walArchiveDir,
                         WALProcedureStore.LeaseRecovery leaseRecovery)
                  throws IOException

Throws:

IOException

Method Detail

start

public void start(int numSlots)
           throws IOException

Description copied from interface: ProcedureStore

Start/Open the procedure store

Parameters:

numSlots - number of threads to be used by the procedure store

Throws:

IOException

stop

public void stop(boolean abort)

Description copied from interface: ProcedureStore

Stop/Close the procedure store

Parameters:

abort - true if the stop is an abort

getNumThreads

public int getNumThreads()

Returns:

the number of threads/slots passed to start()

setRunningProcedureCount

public int setRunningProcedureCount(int count)

Description copied from interface: ProcedureStore

Set the number of procedure running. This can be used, for example, by the store to know how long to wait before a sync.

Returns:

how many procedures are running (may not be same as count).

getStoreTracker

public ProcedureStoreTracker getStoreTracker()

getActiveLogs

public ArrayList<ProcedureWALFile> getActiveLogs()

getCorruptedLogs

public Set<ProcedureWALFile> getCorruptedLogs()

recoverLease

public void recoverLease()
                  throws IOException

Description copied from interface: ProcedureStore

Acquire the lease for the procedure store.

Throws:

IOException

load

public void load(ProcedureStore.ProcedureLoader loader)
          throws IOException

Description copied from interface: ProcedureStore

Load the Procedures in the store.

Parameters:

loader - the ProcedureLoader that will handle the store-load events

Throws:

IOException

insert

public void insert(Procedure<?> proc,
                   Procedure<?>[] subprocs)

Description copied from interface: ProcedureStore

When a procedure is submitted to the executor insert(proc, null) will be called. 'proc' has a 'RUNNABLE' state and the initial information required to start up. When a procedure is executed and it returns children insert(proc, subprocs) will be called. 'proc' has a 'WAITING' state and an update state. 'subprocs' are the children in 'RUNNABLE' state with the initial information.

Parameters:

proc - the procedure to serialize and write to the store.

subprocs - the newly created child of the proc.

insert

public void insert(Procedure<?>[] procs)

Description copied from interface: ProcedureStore

Serialize a set of new procedures. These procedures are freshly submitted to the executor and each procedure has a 'RUNNABLE' state and the initial information required to start up.

Parameters:

procs - the procedures to serialize and write to the store.

update

public void update(Procedure<?> proc)

Description copied from interface: ProcedureStore

The specified procedure was executed, and the new state should be written to the store.

Parameters:

proc - the procedure to serialize and write to the store.

delete

public void delete(long procId)

Description copied from interface: ProcedureStore

The specified procId was removed from the executor, due to completion, abort or failure. The store implementor should remove all the information about the specified procId.

Parameters:

procId - the ID of the procedure to remove.

delete

public void delete(Procedure<?> proc,
                   long[] subProcIds)

Description copied from interface: ProcedureStore

The parent procedure completed. Update the state and mark all the child deleted.

Parameters:

proc - the parent procedure to serialize and write to the store.

subProcIds - the IDs of the sub-procedure to remove.

delete

public void delete(long[] procIds,
                   int offset,
                   int count)

Description copied from interface: ProcedureStore

The specified procIds were removed from the executor, due to completion, abort or failure. The store implementor should remove all the information about the specified procIds.

Parameters:

procIds - the IDs of the procedures to remove.

offset - the array offset from where to start to delete

count - the number of IDs to delete

getSyncMetrics

public ArrayList<WALProcedureStore.SyncMetrics> getSyncMetrics()

syncSlots

protected long syncSlots(org.apache.hadoop.fs.FSDataOutputStream stream,
                         ByteSlot[] slots,
                         int offset,
                         int count)
                  throws IOException

Throws:

IOException

syncStream

protected void syncStream(org.apache.hadoop.fs.FSDataOutputStream stream)
                   throws IOException

Throws:

IOException

getMillisToNextPeriodicRoll

public long getMillisToNextPeriodicRoll()

getMillisFromLastRoll

public long getMillisFromLastRoll()

rollWriterForTesting

public boolean rollWriterForTesting()
                             throws IOException

Throws:

IOException

getWALDir

public org.apache.hadoop.fs.Path getWALDir()

getFileSystem

public org.apache.hadoop.fs.FileSystem getFileSystem()

getLogFilePath

protected org.apache.hadoop.fs.Path getLogFilePath(long logId)
                                            throws IOException

Throws:

IOException

main

public static void main(String[] args)
                 throws IOException

Parses a directory of WALs building up ProcedureState. For testing parse and profiling.

Parameters:

args - Include pointer to directory of WAL files for a store instance to parse & load.

Throws:

IOException