org.apache.hadoop.registry.client.impl.zk

Class CuratorService

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.service.CompositeService

org.apache.hadoop.registry.client.impl.zk.CuratorService

All Implemented Interfaces:

Closeable, AutoCloseable, RegistryConstants, RegistryBindingSource, Service

Direct Known Subclasses:

RegistryOperationsService

@InterfaceAudience.Private
 @InterfaceStability.Evolving
public class CuratorService
extends CompositeService
implements RegistryConstants, RegistryBindingSource

This service binds to Zookeeper via Apache Curator. It is more generic than just the YARN service registry; it does not implement any of the Registry Operations API.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.service.CompositeService

CompositeService.CompositeServiceShutdownHook

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

Service.STATE

Field Summary

Fields inherited from class org.apache.hadoop.service.CompositeService

STOP_ONLY_STARTED_SERVICES

Fields inherited from interface org.apache.hadoop.registry.client.api.RegistryConstants

DEFAULT_REGISTRY_CLIENT_JAAS_CONTEXT, DEFAULT_REGISTRY_ENABLED, DEFAULT_REGISTRY_SECURE, DEFAULT_REGISTRY_SYSTEM_ACCOUNTS, DEFAULT_REGISTRY_USER_ACCOUNTS, DEFAULT_REGISTRY_ZK_QUORUM, DEFAULT_ZK_CONNECTION_TIMEOUT, DEFAULT_ZK_REGISTRY_ROOT, DEFAULT_ZK_RETRY_CEILING, DEFAULT_ZK_RETRY_INTERVAL, DEFAULT_ZK_RETRY_TIMES, DEFAULT_ZK_SESSION_TIMEOUT, KEY_REGISTRY_CLIENT_AUTH, KEY_REGISTRY_CLIENT_AUTHENTICATION_ID, KEY_REGISTRY_CLIENT_AUTHENTICATION_PASSWORD, KEY_REGISTRY_CLIENT_JAAS_CONTEXT, KEY_REGISTRY_ENABLED, KEY_REGISTRY_KERBEROS_REALM, KEY_REGISTRY_SECURE, KEY_REGISTRY_SYSTEM_ACCOUNTS, KEY_REGISTRY_USER_ACCOUNTS, KEY_REGISTRY_ZK_CONNECTION_TIMEOUT, KEY_REGISTRY_ZK_QUORUM, KEY_REGISTRY_ZK_RETRY_CEILING, KEY_REGISTRY_ZK_RETRY_INTERVAL, KEY_REGISTRY_ZK_RETRY_TIMES, KEY_REGISTRY_ZK_ROOT, KEY_REGISTRY_ZK_SESSION_TIMEOUT, PATH_SYSTEM_SERVICES, PATH_USER_SERVICES, PATH_USERS, REGISTRY_CLIENT_AUTH_ANONYMOUS, REGISTRY_CLIENT_AUTH_DIGEST, REGISTRY_CLIENT_AUTH_KERBEROS, REGISTRY_PREFIX, SUBPATH_COMPONENTS, ZK_PREFIX

Constructor Summary

Constructors

Constructor and Description
CuratorService(String name) Create an instance using this service as the binding source (i.e.
CuratorService(String name, RegistryBindingSource bindingSource) Construct the service.

Method Summary

Modifier and Type	Method and Description
boolean	addWriteAccessor(String id, String pass) Add a new write access entry for all future write operations.
String	bindingDiagnosticDetails() Get the binding diagnostics
protected String	buildConnectionString() Override point: get the connection string used to connect to the ZK service
protected String	buildSecurityDiagnostics() Build the security diagnostics string
void	clearWriteAccessors() Clear all write accessors
protected void	createEnsembleProvider() Create the ensemble provider for this registry, by invoking RegistryBindingSource.supplyBindingInformation() on the provider stored in bindingSource Sets ensembleProvider to that value; sets connectionDescription to the binding info for use in toString and logging;
protected String	createFullPath(String path) Create a full path from the registry root and the supplied subdir
ZKPathDumper	dumpPath(boolean verbose) Return a path dumper instance which can do a full dump of the registry tree in its toString() operation
protected String	dumpRegistryRobustly(boolean verbose) Diagnostics method to dump a registry robustly.
RegistryBindingSource	getBindingSource() Get the registry binding source ...
protected RegistrySecurity	getRegistrySecurity() Get the registry security helper
boolean	isSecure() Flag to indicate whether or not the registry is secure.
boolean	maybeCreate(String path, org.apache.zookeeper.CreateMode mode, List<org.apache.zookeeper.data.ACL> acl, boolean createParents) Create a path if it does not exist.
protected IOException	operationFailure(String path, String operation, Exception exception) Create an IOE when an operation fails
protected IOException	operationFailure(String path, String operation, Exception exception, List<org.apache.zookeeper.data.ACL> acls) Create an IOE when an operation fails
protected void	serviceInit(Configuration conf) Init the service.
protected void	serviceStart() Start the service.
protected void	serviceStop() Close the ZK connection if it is open
BindingInformation	supplyBindingInformation() Supply the binding information.
String	toString()
void	zkCreate(String path, org.apache.zookeeper.CreateMode mode, byte[] data, List<org.apache.zookeeper.data.ACL> acls) Create a path with given data.
void	zkDelete(String path, boolean recursive, org.apache.curator.framework.api.BackgroundCallback backgroundCallback) Delete a directory/directory tree.
List<org.apache.zookeeper.data.ACL>	zkGetACLS(String path) Get the ACLs of a path
List<String>	zkList(String path) List all children of a path
void	zkMkParentPath(String path, List<org.apache.zookeeper.data.ACL> acl) Recursively make a path
boolean	zkMkPath(String path, org.apache.zookeeper.CreateMode mode, boolean createParents, List<org.apache.zookeeper.data.ACL> acls) Create a directory.
boolean	zkPathExists(String path) Probe for a path existing
String	zkPathMustExist(String path) Verify a path exists
byte[]	zkRead(String path) Read data on a path
boolean	zkSet(String path, org.apache.zookeeper.CreateMode mode, byte[] data, List<org.apache.zookeeper.data.ACL> acl, boolean overwrite) Create or update an entry
org.apache.zookeeper.data.Stat	zkStat(String path) Stat the file
void	zkUpdate(String path, byte[] data) Update the data for a path

Methods inherited from class org.apache.hadoop.service.CompositeService

addIfService, addService, getServices, removeService

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, setConfig, start, stop, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

CuratorService

public CuratorService(String name,
                      RegistryBindingSource bindingSource)

Construct the service.

Parameters:

name - service name

bindingSource - source of binding information. If null: use this instance

CuratorService

public CuratorService(String name)

Create an instance using this service as the binding source (i.e. read configuration options from the registry)

Parameters:

name - service name

Method Detail

serviceInit

protected void serviceInit(Configuration conf)
                    throws Exception

Init the service. This is where the security bindings are set up

Overrides:

serviceInit in class CompositeService

Parameters:

conf - configuration of the service

Throws:

Exception

serviceStart

protected void serviceStart()
                     throws Exception

Start the service. This is where the curator instance is started.

Overrides:

serviceStart in class CompositeService

Throws:

Exception

serviceStop

protected void serviceStop()
                    throws Exception

Close the ZK connection if it is open

Overrides:

serviceStop in class CompositeService

Throws:

Exception

isSecure

public boolean isSecure()

Flag to indicate whether or not the registry is secure. Valid once the service is inited.

Returns:

service security policy

getRegistrySecurity

protected RegistrySecurity getRegistrySecurity()

Get the registry security helper

Returns:

the registry security helper

buildSecurityDiagnostics

protected String buildSecurityDiagnostics()

Build the security diagnostics string

Returns:

a string for diagnostics

toString

public String toString()

Overrides:

toString in class AbstractService

bindingDiagnosticDetails

public String bindingDiagnosticDetails()

Get the binding diagnostics

Returns:

a diagnostics string valid after the service is started.

createFullPath

protected String createFullPath(String path)
                         throws IOException

Create a full path from the registry root and the supplied subdir

Parameters:

path - path of operation

Returns:

an absolute path

Throws:

IllegalArgumentException - if the path is invalide

IOException

getBindingSource

public RegistryBindingSource getBindingSource()

Get the registry binding source ... this can be used to create new ensemble providers

Returns:

the registry binding source in use

createEnsembleProvider

protected void createEnsembleProvider()

Create the ensemble provider for this registry, by invoking RegistryBindingSource.supplyBindingInformation() on the provider stored in bindingSource Sets ensembleProvider to that value; sets connectionDescription to the binding info for use in toString and logging;

supplyBindingInformation

public BindingInformation supplyBindingInformation()

Supply the binding information. This implementation returns a fixed ensemble bonded to the quorum supplied by buildConnectionString()

Specified by:

supplyBindingInformation in interface RegistryBindingSource

Returns:

the binding information

buildConnectionString

protected String buildConnectionString()

Override point: get the connection string used to connect to the ZK service

Returns:

a registry quorum

operationFailure

protected IOException operationFailure(String path,
                                       String operation,
                                       Exception exception)

Create an IOE when an operation fails

Parameters:

path - path of operation

operation - operation attempted

exception - caught the exception caught

Returns:

an IOE to throw that contains the path and operation details.

operationFailure

protected IOException operationFailure(String path,
                                       String operation,
                                       Exception exception,
                                       List<org.apache.zookeeper.data.ACL> acls)

Create an IOE when an operation fails

Parameters:

path - path of operation

operation - operation attempted

exception - caught the exception caught

Returns:

an IOE to throw that contains the path and operation details.

maybeCreate

public boolean maybeCreate(String path,
                           org.apache.zookeeper.CreateMode mode,
                           List<org.apache.zookeeper.data.ACL> acl,
                           boolean createParents)
                    throws IOException

Create a path if it does not exist. The check is poll + create; there's a risk that another process may create the same path before the create() operation is executed/ propagated to the ZK node polled.

Parameters:

path - path to create

acl - ACL for path -used when creating a new entry

createParents - flag to trigger parent creation

Returns:

true iff the path was created

Throws:

IOException

zkStat

public org.apache.zookeeper.data.Stat zkStat(String path)
                                      throws IOException

Stat the file

Parameters:

path - path of operation

Returns:

a curator stat entry

Throws:

IOException - on a failure

PathNotFoundException - if the path was not found

zkGetACLS

public List<org.apache.zookeeper.data.ACL> zkGetACLS(String path)
                                              throws IOException

Get the ACLs of a path

Parameters:

path - path of operation

Returns:

a possibly empty list of ACLs

Throws:

IOException

zkPathExists

public boolean zkPathExists(String path)
                     throws IOException

Probe for a path existing

Parameters:

path - path of operation

Returns:

true if the path was visible from the ZK server queried.

Throws:

IOException - on any exception other than PathNotFoundException

zkPathMustExist

public String zkPathMustExist(String path)
                       throws IOException

Verify a path exists

Parameters:

path - path of operation

Throws:

PathNotFoundException - if the path is absent

IOException

zkMkPath

public boolean zkMkPath(String path,
                        org.apache.zookeeper.CreateMode mode,
                        boolean createParents,
                        List<org.apache.zookeeper.data.ACL> acls)
                 throws IOException

Create a directory. It is not an error if it already exists

Parameters:

path - path to create

mode - mode for path

createParents - flag to trigger parent creation

acls - ACL for path

Throws:

IOException - any problem

zkMkParentPath

public void zkMkParentPath(String path,
                           List<org.apache.zookeeper.data.ACL> acl)
                    throws IOException

Recursively make a path

Parameters:

path - path to create

acl - ACL for path

Throws:

IOException - any problem

zkCreate

public void zkCreate(String path,
                     org.apache.zookeeper.CreateMode mode,
                     byte[] data,
                     List<org.apache.zookeeper.data.ACL> acls)
              throws IOException

Create a path with given data. byte[0] is used for a path without data

Parameters:

path - path of operation

data - initial data

acls -

Throws:

IOException

zkUpdate

public void zkUpdate(String path,
                     byte[] data)
              throws IOException

Update the data for a path

Parameters:

path - path of operation

data - new data

Throws:

IOException

zkSet

public boolean zkSet(String path,
                     org.apache.zookeeper.CreateMode mode,
                     byte[] data,
                     List<org.apache.zookeeper.data.ACL> acl,
                     boolean overwrite)
              throws IOException

Create or update an entry

Parameters:

path - path

data - data

acl - ACL for path -used when creating a new entry

overwrite - enable overwrite

Returns:

true if the entry was created, false if it was simply updated.

Throws:

IOException

zkDelete

public void zkDelete(String path,
                     boolean recursive,
                     org.apache.curator.framework.api.BackgroundCallback backgroundCallback)
              throws IOException

Delete a directory/directory tree. It is not an error to delete a path that does not exist

Parameters:

path - path of operation

recursive - flag to trigger recursive deletion

backgroundCallback - callback; this being set converts the operation into an async/background operation. task

Throws:

IOException - on problems other than no-such-path

zkList

public List<String> zkList(String path)
                    throws IOException

List all children of a path

Parameters:

path - path of operation

Returns:

a possibly empty list of children

Throws:

IOException

zkRead

public byte[] zkRead(String path)
              throws IOException

Read data on a path

Parameters:

path - path of operation

Returns:

the data

Throws:

IOException - read failure

dumpPath

public ZKPathDumper dumpPath(boolean verbose)

Return a path dumper instance which can do a full dump of the registry tree in its toString() operation

Parameters:

verbose - verbose flag - includes more details (such as ACLs)

Returns:

a class to dump the registry

addWriteAccessor

public boolean addWriteAccessor(String id,
                                String pass)
                         throws IOException

Add a new write access entry for all future write operations.

Parameters:

id - ID to use

pass - password

Throws:

IOException - on any failure to build the digest

clearWriteAccessors

public void clearWriteAccessors()

Clear all write accessors

dumpRegistryRobustly

protected String dumpRegistryRobustly(boolean verbose)

Diagnostics method to dump a registry robustly. Any exception raised is swallowed

Parameters:

verbose - verbose path dump

Returns:

the registry tree