Package org.pipecraft.infra.storage.amazon_s3

Class S3Bucket

java.lang.Object

org.pipecraft.infra.storage.Bucket<com.amazonaws.services.s3.model.S3ObjectSummary>

org.pipecraft.infra.storage.amazon_s3.S3Bucket

public class S3Bucket
extends org.pipecraft.infra.storage.Bucket<com.amazonaws.services.s3.model.S3ObjectSummary>

A storage bucket implementation based on Amazon's S3. Supports the optional operation of uploading files with public access, but doesn't support the following optional Bucket operations: - Generating read/write signed URLs - Composing remote files - Exclusive file creation operation (lock functionality) - Uploading data using OutputStream In addition S3 doesn't support List-after-write consistency.

Author:

Ben Bonfil , Eyal Schneider

Field Summary

Fields inherited from class org.pipecraft.infra.storage.Bucket

DEFAULT_RETRIER, DEFAULT_RETRY_INITIAL_SLEEP_SEC, DEFAULT_RETRY_MAX_ATTEMPTS, DEFAULT_RETRY_WAIT_TIME_FACTOR, DONE_FILE_NAME

Method Summary

Modifier and Type	Method	Description
com.amazonaws.services.s3.model.S3ObjectSummary	compose(List<String> gsPaths, String composedFilePath, boolean removeComprisingFiles)
void	copyToAnotherBucket(String fromKey, String toBucket, String toKey)
void	delete(com.amazonaws.services.s3.model.S3ObjectSummary obj)
boolean	exists(String key)
URL	generateReadOnlyUrl(String key, int expirationSeconds)
URL	generateResumableSignedUrlForUpload(String key, String contentType, int expirationSeconds, Long maxContentLengthInBytes, boolean isPublic)
URL	generateSignedUrl(String key, String contentType, int expirationSeconds, boolean isPublicRead)
void	get(com.amazonaws.services.s3.model.S3ObjectSummary meta, File output)
org.pipecraft.infra.io.SizedInputStream	getAsStream(com.amazonaws.services.s3.model.S3ObjectSummary meta, int chunkSize)
Long	getLastUpdated(com.amazonaws.services.s3.model.S3ObjectSummary objMetadata)
long	getLength(com.amazonaws.services.s3.model.S3ObjectSummary keyMetadata)
com.amazonaws.services.s3.model.S3ObjectSummary	getObjectMetadata(String key)
OutputStream	getOutputStream(String key, int chunkSize)
String	getPath(com.amazonaws.services.s3.model.S3ObjectSummary keyMetadata)
Iterator<com.amazonaws.services.s3.model.S3ObjectSummary>	listObjects(String folderPath, boolean recursive)
void	put(String key, InputStream input, long length, String contentType, boolean isPublic, boolean allowOverride)

Methods inherited from class org.pipecraft.infra.storage.Bucket

copy, copyFolderRecursiveInterruptibly, copyFolderRecursiveInterruptibly, copyInterruptibly, copyInterruptibly, copyToAnotherBucketInterruptibly, copyToAnotherBucketInterruptibly, delete, deleteAllByMetaInterruptibly, deleteAllByMetaInterruptibly, deleteAllByMetaInterruptibly, deleteAllByMetaInterruptibly, deleteAllInterruptibly, deleteAllInterruptibly, deleteAllInterruptibly, deleteAllInterruptibly, deleteFolderRecursiveInterruptibly, deleteFolderRecursiveInterruptibly, deleteFolderRegularFiles, deleteInterruptibly, deleteInterruptibly, deleteInterruptibly, deleteInterruptibly, generateResumableSignedUrlForUpload, generateSignedUrl, get, getAllRegularFiles, getAllRegularFilesByMetaInterruptibly, getAllRegularFilesByMetaInterruptibly, getAllRegularFilesByMetaInterruptibly, getAllRegularFilesByMetaInterruptibly, getAllRegularFilesInterruptibly, getAllRegularFilesInterruptibly, getAllRegularFilesInterruptibly, getAllRegularFilesInterruptibly, getAllRegularFilesInterruptibly, getAllRegularFilesInterruptibly, getAllRegularFilesInterruptibly, getAsStream, getAsStream, getAsStream, getBucketName, getFromJson, getInterruptibly, getInterruptibly, getInterruptibly, getInterruptibly, getLastFile, getObjectMetadata, getObjectMetadata, getOutputStream, getSliced, getSliced, isFile, isFilePath, isFolderPath, listFiles, listFiles, listFiles, listFilesRecursive, listFilesRecursive, listFilesRecursive, listFolders, listObjects, listObjects, moveFolderRecursive, moveFolderRecursive, moveInterruptibly, moveInterruptibly, normalizeFolderPath, put, putAllInterruptibly, putAllInterruptibly, putAllRecursiveInterruptibly, putAllRecursiveInterruptibly, putDoneFile, putEmptyFile, putFile, putFileInterruptibly, putFileInterruptibly, putLockFile, putPrivate, putPrivate, putPublic, putPublic, putUniquePrivate, putUniquePrivate, putUniquePrivate, putUniquePublic, putUniquePublic, putUniquePublic, validateNotFolderPath

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

put

public void put(String key,
                InputStream input,
                long length,
                String contentType,
                boolean isPublic,
                boolean allowOverride)
         throws IOException

Specified by:

put in class org.pipecraft.infra.storage.Bucket<com.amazonaws.services.s3.model.S3ObjectSummary>

Throws:

IOException

getOutputStream

public OutputStream getOutputStream(String key,
                                    int chunkSize)
                             throws IOException

Specified by:

getOutputStream in class org.pipecraft.infra.storage.Bucket<com.amazonaws.services.s3.model.S3ObjectSummary>

Throws:

IOException

copyToAnotherBucket

public void copyToAnotherBucket(String fromKey,
                                String toBucket,
                                String toKey)
                         throws IOException

Specified by:

copyToAnotherBucket in class org.pipecraft.infra.storage.Bucket<com.amazonaws.services.s3.model.S3ObjectSummary>

Throws:

IOException

delete

public void delete(com.amazonaws.services.s3.model.S3ObjectSummary obj)
            throws IOException

Specified by:

delete in class org.pipecraft.infra.storage.Bucket<com.amazonaws.services.s3.model.S3ObjectSummary>

Throws:

IOException

get

public void get(com.amazonaws.services.s3.model.S3ObjectSummary meta,
                File output)
         throws IOException

Specified by:

get in class org.pipecraft.infra.storage.Bucket<com.amazonaws.services.s3.model.S3ObjectSummary>

Throws:

IOException

getAsStream

public org.pipecraft.infra.io.SizedInputStream getAsStream(com.amazonaws.services.s3.model.S3ObjectSummary meta,
                                                           int chunkSize)
                                                    throws IOException

Specified by:

getAsStream in class org.pipecraft.infra.storage.Bucket<com.amazonaws.services.s3.model.S3ObjectSummary>

Throws:

IOException

listObjects

public Iterator<com.amazonaws.services.s3.model.S3ObjectSummary> listObjects(String folderPath,
                                                                             boolean recursive)
                                                                      throws IOException

Specified by:

listObjects in class org.pipecraft.infra.storage.Bucket<com.amazonaws.services.s3.model.S3ObjectSummary>

Throws:

IOException

generateSignedUrl

public URL generateSignedUrl(String key,
                             String contentType,
                             int expirationSeconds,
                             boolean isPublicRead)

Specified by:

generateSignedUrl in class org.pipecraft.infra.storage.Bucket<com.amazonaws.services.s3.model.S3ObjectSummary>

generateReadOnlyUrl

public URL generateReadOnlyUrl(String key,
                               int expirationSeconds)

Specified by:

generateReadOnlyUrl in class org.pipecraft.infra.storage.Bucket<com.amazonaws.services.s3.model.S3ObjectSummary>

generateResumableSignedUrlForUpload

public URL generateResumableSignedUrlForUpload(String key,
                                               String contentType,
                                               int expirationSeconds,
                                               Long maxContentLengthInBytes,
                                               boolean isPublic)

Specified by:

generateResumableSignedUrlForUpload in class org.pipecraft.infra.storage.Bucket<com.amazonaws.services.s3.model.S3ObjectSummary>

getObjectMetadata

public com.amazonaws.services.s3.model.S3ObjectSummary getObjectMetadata(String key)
                                                                  throws IOException

Specified by:

getObjectMetadata in class org.pipecraft.infra.storage.Bucket<com.amazonaws.services.s3.model.S3ObjectSummary>

Throws:

IOException

getPath

public String getPath(com.amazonaws.services.s3.model.S3ObjectSummary keyMetadata)

Specified by:

getPath in class org.pipecraft.infra.storage.Bucket<com.amazonaws.services.s3.model.S3ObjectSummary>

getLength

public long getLength(com.amazonaws.services.s3.model.S3ObjectSummary keyMetadata)

Specified by:

getLength in class org.pipecraft.infra.storage.Bucket<com.amazonaws.services.s3.model.S3ObjectSummary>

getLastUpdated

public Long getLastUpdated(com.amazonaws.services.s3.model.S3ObjectSummary objMetadata)

Specified by:

getLastUpdated in class org.pipecraft.infra.storage.Bucket<com.amazonaws.services.s3.model.S3ObjectSummary>

exists

public boolean exists(String key)
               throws IOException

Specified by:

exists in class org.pipecraft.infra.storage.Bucket<com.amazonaws.services.s3.model.S3ObjectSummary>

Throws:

IOException

compose

public com.amazonaws.services.s3.model.S3ObjectSummary compose(List<String> gsPaths,
                                                               String composedFilePath,
                                                               boolean removeComprisingFiles)
                                                        throws IOException

Specified by:

compose in class org.pipecraft.infra.storage.Bucket<com.amazonaws.services.s3.model.S3ObjectSummary>

Throws:

IOException