Package alluxio.underfs.gcs

Class GCSUnderFileSystem

java.lang.Object

alluxio.underfs.BaseUnderFileSystem

alluxio.underfs.ObjectUnderFileSystem

alluxio.underfs.gcs.GCSUnderFileSystem

All Implemented Interfaces:

alluxio.underfs.UnderFileSystem, java.io.Closeable, java.lang.AutoCloseable

@ThreadSafe
public class GCSUnderFileSystem
extends alluxio.underfs.ObjectUnderFileSystem

Google Cloud Storage UnderFileSystem implementation based on the jets3t library.

Nested Class Summary

Nested classes/interfaces inherited from class alluxio.underfs.ObjectUnderFileSystem

alluxio.underfs.ObjectUnderFileSystem.DeleteBuffer, alluxio.underfs.ObjectUnderFileSystem.ObjectListingChunk, alluxio.underfs.ObjectUnderFileSystem.ObjectPermissions, alluxio.underfs.ObjectUnderFileSystem.ObjectStatus, alluxio.underfs.ObjectUnderFileSystem.ObjectStoreOperation<T extends java.lang.Object>, alluxio.underfs.ObjectUnderFileSystem.OperationBuffer<T extends java.lang.Object>, alluxio.underfs.ObjectUnderFileSystem.RenameBuffer

Nested classes/interfaces inherited from interface alluxio.underfs.UnderFileSystem

alluxio.underfs.UnderFileSystem.Factory, alluxio.underfs.UnderFileSystem.SpaceType

Field Summary

Fields inherited from class alluxio.underfs.ObjectUnderFileSystem

mExecutorService, mRootKeySupplier, PATH_SEPARATOR, PATH_SEPARATOR_CHAR

Fields inherited from class alluxio.underfs.BaseUnderFileSystem

EMPTY_ACL, mUfsConf, mUri

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	GCSUnderFileSystem(alluxio.AlluxioURI uri, org.jets3t.service.impl.rest.httpclient.GoogleStorageService googleStorageService, java.lang.String bucketName, alluxio.underfs.UnderFileSystemConfiguration conf)	Constructor for GCSUnderFileSystem.

Method Summary

Modifier and Type	Method	Description
protected boolean	copyObject(java.lang.String src, java.lang.String dst)
boolean	createEmptyObject(java.lang.String key)
static GCSUnderFileSystem	createInstance(alluxio.AlluxioURI uri, alluxio.underfs.UnderFileSystemConfiguration conf)	Constructs a new instance of GCSUnderFileSystem.
protected java.io.OutputStream	createObject(java.lang.String key)
protected boolean	deleteObject(java.lang.String key)
alluxio.underfs.UfsDirectoryStatus	getExistingDirectoryStatus(java.lang.String path)
protected java.lang.String	getFolderSuffix()
protected alluxio.underfs.ObjectUnderFileSystem.ObjectListingChunk	getObjectListingChunk(java.lang.String key, boolean recursive)
protected alluxio.underfs.ObjectUnderFileSystem.ObjectStatus	getObjectStatus(java.lang.String key)
protected alluxio.underfs.ObjectUnderFileSystem.ObjectPermissions	getPermissions()
protected java.lang.String	getRootKey()
java.lang.String	getUnderFSType()
java.io.InputStream	openExistingFile(java.lang.String path)
java.io.InputStream	openExistingFile(java.lang.String path, alluxio.underfs.options.OpenOptions options)
protected java.io.InputStream	openObject(java.lang.String key, alluxio.underfs.options.OpenOptions options, alluxio.retry.RetryPolicy retryPolicy)
alluxio.PositionReader	openPositionRead(java.lang.String path, long fileLength)
void	setMode(java.lang.String path, short mode)
void	setOwner(java.lang.String path, java.lang.String user, java.lang.String group)

Methods inherited from class alluxio.underfs.ObjectUnderFileSystem

cleanup, close, connectFromMaster, connectFromWorker, convertToFolderName, create, createNonexistingFile, createNonexistingFile, deleteDirectory, deleteExistingDirectory, deleteExistingDirectory, deleteExistingFile, deleteFile, deleteObjects, getBlockSizeByte, getChildName, getDirectoryStatus, getExistingFileStatus, getExistingStatus, getFileLocations, getFileLocations, getFileStatus, getListingChunkLength, getListingChunkLengthMax, getObjectListingChunkForPath, getParentPath, getSpace, getStatus, isDirectory, isExistingDirectory, isFile, isObjectStorage, isRoot, listInternal, listStatus, listStatus, mkdirs, mkdirsInternal, open, parentExists, renameDirectory, renameFile, renameRenamableDirectory, renameRenamableFile, retryOnException, stripPrefixIfPresent, supportsFlush

Methods inherited from class alluxio.underfs.BaseUnderFileSystem

create, deleteDirectory, exists, getAclPair, getActiveSyncInfo, getConfiguration, getFingerprint, getOperationMode, getParsedFingerprint, getPhysicalStores, isSeekable, mkdirs, open, resolveUri, setAclEntries, startActiveSyncPolling, startSync, stopActiveSyncPolling, stopSync, supportsActiveSync, validatePath

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface alluxio.underfs.UnderFileSystem

listStatuses

Constructor Detail

GCSUnderFileSystem

protected GCSUnderFileSystem(alluxio.AlluxioURI uri,
                             org.jets3t.service.impl.rest.httpclient.GoogleStorageService googleStorageService,
                             java.lang.String bucketName,
                             alluxio.underfs.UnderFileSystemConfiguration conf)

Constructor for GCSUnderFileSystem.

Parameters:

uri - the AlluxioURI for this UFS

googleStorageService - the Jets3t GCS client

bucketName - bucket name of user's configured Alluxio bucket

conf - configuration for this UFS

Method Detail

createInstance

public static GCSUnderFileSystem createInstance(alluxio.AlluxioURI uri,
                                                alluxio.underfs.UnderFileSystemConfiguration conf)
                                         throws org.jets3t.service.ServiceException

Constructs a new instance of GCSUnderFileSystem.

Parameters:

uri - the AlluxioURI for this UFS

conf - the configuration for this UFS

Returns:

the created GCSUnderFileSystem instance

Throws:

org.jets3t.service.ServiceException - when a connection to GCS could not be created

getUnderFSType

public java.lang.String getUnderFSType()

setOwner

public void setOwner(java.lang.String path,
                     java.lang.String user,
                     java.lang.String group)

setMode

public void setMode(java.lang.String path,
                    short mode)
             throws java.io.IOException

Throws:

java.io.IOException

getExistingDirectoryStatus

public alluxio.underfs.UfsDirectoryStatus getExistingDirectoryStatus(java.lang.String path)
                                                              throws java.io.IOException

Specified by:

getExistingDirectoryStatus in interface alluxio.underfs.UnderFileSystem

Overrides:

getExistingDirectoryStatus in class alluxio.underfs.ObjectUnderFileSystem

Throws:

java.io.IOException

openExistingFile

public java.io.InputStream openExistingFile(java.lang.String path)
                                     throws java.io.IOException

Specified by:

openExistingFile in interface alluxio.underfs.UnderFileSystem

Overrides:

openExistingFile in class alluxio.underfs.ObjectUnderFileSystem

Throws:

java.io.IOException

openExistingFile

public java.io.InputStream openExistingFile(java.lang.String path,
                                            alluxio.underfs.options.OpenOptions options)
                                     throws java.io.IOException

Specified by:

openExistingFile in interface alluxio.underfs.UnderFileSystem

Overrides:

openExistingFile in class alluxio.underfs.ObjectUnderFileSystem

Throws:

java.io.IOException

openPositionRead

public alluxio.PositionReader openPositionRead(java.lang.String path,
                                               long fileLength)

copyObject

protected boolean copyObject(java.lang.String src,
                             java.lang.String dst)

Specified by:

copyObject in class alluxio.underfs.ObjectUnderFileSystem

createEmptyObject

public boolean createEmptyObject(java.lang.String key)

Specified by:

createEmptyObject in class alluxio.underfs.ObjectUnderFileSystem

createObject

protected java.io.OutputStream createObject(java.lang.String key)
                                     throws java.io.IOException

Specified by:

createObject in class alluxio.underfs.ObjectUnderFileSystem

Throws:

java.io.IOException

deleteObject

protected boolean deleteObject(java.lang.String key)
                        throws java.io.IOException

Specified by:

deleteObject in class alluxio.underfs.ObjectUnderFileSystem

Throws:

java.io.IOException

getFolderSuffix

protected java.lang.String getFolderSuffix()

Specified by:

getFolderSuffix in class alluxio.underfs.ObjectUnderFileSystem

getObjectListingChunk

protected alluxio.underfs.ObjectUnderFileSystem.ObjectListingChunk getObjectListingChunk(java.lang.String key,
                                                                                         boolean recursive)
                                                                                  throws java.io.IOException

Specified by:

getObjectListingChunk in class alluxio.underfs.ObjectUnderFileSystem

Throws:

java.io.IOException

getObjectStatus

protected alluxio.underfs.ObjectUnderFileSystem.ObjectStatus getObjectStatus(java.lang.String key)
                                                                      throws java.io.IOException

Specified by:

getObjectStatus in class alluxio.underfs.ObjectUnderFileSystem

Throws:

java.io.IOException

getPermissions

protected alluxio.underfs.ObjectUnderFileSystem.ObjectPermissions getPermissions()

Specified by:

getPermissions in class alluxio.underfs.ObjectUnderFileSystem

getRootKey

protected java.lang.String getRootKey()

Specified by:

getRootKey in class alluxio.underfs.ObjectUnderFileSystem

openObject

protected java.io.InputStream openObject(java.lang.String key,
                                         alluxio.underfs.options.OpenOptions options,
                                         alluxio.retry.RetryPolicy retryPolicy)

Specified by:

openObject in class alluxio.underfs.ObjectUnderFileSystem