Package org.duracloud.s3storage

Class S3StorageProvider

java.lang.Object
org.duracloud.storage.provider.StorageProviderBase
org.duracloud.s3storage.S3StorageProvider
All Implemented Interfaces:
StorageProvider
public class S3StorageProvider extends StorageProviderBase
Provides content storage backed by Amazon's Simple Storage Service.
Author:
Bill Branan

  • Field Details

    • MAX_ITEM_COUNT

      protected static final int MAX_ITEM_COUNT
      See Also:

    • DEFAULT_STORAGE_CLASS

      protected static final com.amazonaws.services.s3.model.StorageClass DEFAULT_STORAGE_CLASS

    • HIDDEN_SPACE_PREFIX

      public static final String HIDDEN_SPACE_PREFIX
      See Also:

    • HEADER_VALUE_PREFIX

      protected static final String HEADER_VALUE_PREFIX

    • HEADER_KEY_SUFFIX

      protected static final String HEADER_KEY_SUFFIX
      See Also:

    • accessKeyId

      protected String accessKeyId

    • s3Client

      protected com.amazonaws.services.s3.AmazonS3 s3Client

  • Constructor Details

    • S3StorageProvider

      public S3StorageProvider(String accessKey, String secretKey)

    • S3StorageProvider

      public S3StorageProvider(String accessKey, String secretKey, Map<String,String> options)

    • S3StorageProvider

      public S3StorageProvider(com.amazonaws.services.s3.AmazonS3 s3Client, String accessKey, Map<String,String> options)

  • Method Details

    • getStorageProviderType

      public StorageProviderType getStorageProviderType()

    • getSpaces

      public Iterator<String> getSpaces()

    • getSpaceContents

      public Iterator<String> getSpaceContents(String spaceId, String prefix)

    • getSpaceContentsChunked

      public List<String> getSpaceContentsChunked(String spaceId, String prefix, long maxResults, String marker)

    • spaceExists

      protected boolean spaceExists(String spaceId)
      Specified by:
      spaceExists in class StorageProviderBase

    • createSpace

      public void createSpace(String spaceId)

    • createBucket

      protected com.amazonaws.services.s3.model.Bucket createBucket(String spaceId)

    • getHiddenBucketName

      protected String getHiddenBucketName(String spaceId)

    • createHiddenSpace

      public String createHiddenSpace(String spaceId, int expirationInDays)
      Creates a "hidden" space. This space will not be returned by the StorageProvider.getSpaces() method. It can be accessed using the getSpace* methods. You must know the name of the space in order to access it.
      Parameters:
      spaceId - The spaceId
      expirationInDays - The number of days before content in the space is automatically deleted.
      Returns:

    • getStoragePolicy

      protected StoragePolicy getStoragePolicy()
      Defines the storage policy for the primary S3 provider. Subclasses can define different policy choices.
      Returns:
      storage policy to set, or null if no policy should be defined

    • setSpaceLifecycle

      public void setSpaceLifecycle(String bucketName, com.amazonaws.services.s3.model.BucketLifecycleConfiguration config)
      Sets a lifecycle policy on an S3 bucket based on the given configuration
      Parameters:
      bucketName - name of the bucket to update
      config - bucket lifecycle configuration

    • getNewBucketName

      protected String getNewBucketName(String spaceId)

    • formattedDate

      protected String formattedDate(Date date)

    • removeSpace

      public void removeSpace(String spaceId)
      Specified by:
      removeSpace in class StorageProviderBase

    • getAllSpaceProperties

      protected Map<String,String> getAllSpaceProperties(String spaceId)
      Specified by:
      getAllSpaceProperties in class StorageProviderBase

    • getSpaceCount

      protected String getSpaceCount(String spaceId, int maxCount)

    • getBucketCreationDate

      protected String getBucketCreationDate(String bucketName)

    • doSetSpaceProperties

      protected void doSetSpaceProperties(String spaceId, Map<String,String> spaceProperties)
      Specified by:
      doSetSpaceProperties in class StorageProviderBase

    • replaceInMapValues

      protected Map<String,String> replaceInMapValues(Map<String,String> map, String oldVal, String newVal)

    • addHiddenContent

      public String addHiddenContent(String spaceId, String contentId, String contentMimeType, InputStream content)
      Adds content to a hidden space.
      Parameters:
      spaceId - hidden spaceId
      contentId -
      contentMimeType -
      content -
      Returns:

    • addContent

      public String addContent(String spaceId, String contentId, String contentMimeType, Map<String,String> userProperties, long contentSize, String contentChecksum, InputStream content)

    • doesContentExistWithExpectedChecksum

      protected String doesContentExistWithExpectedChecksum(String bucketName, String contentId, String expectedChecksum)

    • wait

      protected void wait(int seconds)

    • copyContent

      public String copyContent(String sourceSpaceId, String sourceContentId, String destSpaceId, String destContentId)

    • getContent

      public RetrievedContent getContent(String spaceId, String contentId)

    • getContent

      public RetrievedContent getContent(String spaceId, String contentId, String range)

    • deleteContent

      public void deleteContent(String spaceId, String contentId)

    • setContentProperties

      public void setContentProperties(String spaceId, String contentId, Map<String,String> contentProperties)

    • removeCalculatedProperties

      protected Map<String,String> removeCalculatedProperties(Map<String,String> contentProperties)
      Overrides:
      removeCalculatedProperties in class StorageProviderBase

    • throwIfContentNotExist

      protected void throwIfContentNotExist(String bucketName, String contentId)

    • updateObjectProperties

      protected void updateObjectProperties(String bucketName, String contentId, com.amazonaws.services.s3.model.ObjectMetadata objMetadata)

    • getContentProperties

      public Map<String,String> getContentProperties(String spaceId, String contentId)

    • getSpaceProperties

      public Map<String,String> getSpaceProperties(String spaceId)
      Specified by:
      getSpaceProperties in interface StorageProvider
      Overrides:
      getSpaceProperties in class StorageProviderBase

    • prepContentProperties

      protected Map<String,String> prepContentProperties(com.amazonaws.services.s3.model.ObjectMetadata objMetadata)

    • getETagValue

      protected String getETagValue(String etag)

    • getBucketName

      public String getBucketName(String spaceId)
      Gets the name of an existing bucket based on a space ID. If no bucket with this spaceId exists, throws a NotFoundException
      Parameters:
      spaceId - the space Id to convert into an S3 bucket name
      Returns:
      S3 bucket name of a given DuraCloud space
      Throws:
      NotFoundException - if no bucket matches this spaceID

    • getSpaceId

      protected String getSpaceId(String bucketName)
      Converts a bucket name into what could be passed in as a space ID.
      Parameters:
      bucketName - name of the S3 bucket
      Returns:
      the DuraCloud space name equivalent to a given S3 bucket Id

    • isSpace

      protected boolean isSpace(String bucketName)
      Determines if an S3 bucket is a DuraCloud space
      Parameters:
      bucketName - name of the S3 bucket
      Returns:
      true if the given S3 bucket name is named according to the DuraCloud space naming conventions, false otherwise

    • getSpaceFree

      protected String getSpaceFree(String name)
      Replaces all spaces with "%20"
      Parameters:
      name - string with possible space
      Returns:
      converted to string without spaces

    • getWithSpace

      protected String getWithSpace(String name)
      Converts "%20" back to spaces
      Parameters:
      name - string
      Returns:
      converted to spaces

    • encodeHeaderValue

      protected static String encodeHeaderValue(String userMetaValue)
      Ensures compliance with https://tools.ietf.org/html/rfc5987#section-3.2.2
      Parameters:
      userMetaValue -
      Returns:

    • decodeHeaderValue

      protected static String decodeHeaderValue(String userMetaValue)

    • encodeHeaderKey

      protected static String encodeHeaderKey(String userMetaName)
      Ensures compliance with https://tools.ietf.org/html/rfc5987#section-3.2.2
      Parameters:
      userMetaName -
      Returns:

    • decodeHeaderKey

      protected static String decodeHeaderKey(String userMetaName)