org.apache.hadoop.fs.aliyun.oss

Class AliyunOSSUtils

java.lang.Object

org.apache.hadoop.fs.aliyun.oss.AliyunOSSUtils

public final class AliyunOSSUtils
extends Object

Utility methods for Aliyun OSS code.

Method Summary

Modifier and Type	Method and Description
static long	calculatePartSize(long contentLength, long minPartSize) Calculate a proper size of multipart piece.
static File	createTmpFileForWrite(String path, long size, org.apache.hadoop.conf.Configuration conf) Demand create the directory allocator, then create a temporary file.
static com.aliyun.oss.common.auth.CredentialsProvider	getCredentialsProvider(URI uri, org.apache.hadoop.conf.Configuration conf) Create credential provider specified by configuration, or create default credential provider if not specified.
static long	getMultipartSizeProperty(org.apache.hadoop.conf.Configuration conf, String property, long defVal) Get a size property from the configuration: this property must be at least equal to Constants.MULTIPART_MIN_SIZE.
static String	getValueWithKey(org.apache.hadoop.conf.Configuration conf, String key) Used to get password from configuration.
static int	intPositiveOption(org.apache.hadoop.conf.Configuration conf, String key, int defVal)
static String	maybeAddTrailingSlash(String key) Turns a path (relative or otherwise) into an OSS key, adding a trailing "/" if the path is not the root and does not already have a "/" at the end.
static boolean	objectRepresentsDirectory(String name, long size) Check if OSS object represents a directory.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

intPositiveOption

public static int intPositiveOption(org.apache.hadoop.conf.Configuration conf,
                                    String key,
                                    int defVal)

getValueWithKey

public static String getValueWithKey(org.apache.hadoop.conf.Configuration conf,
                                     String key)
                              throws IOException

Used to get password from configuration.

Parameters:

conf - configuration that contains password information

key - the key of the password

Returns:

the value for the key

Throws:

IOException - if failed to get password from configuration

calculatePartSize

public static long calculatePartSize(long contentLength,
                                     long minPartSize)

Calculate a proper size of multipart piece. If minPartSize is too small, the number of multipart pieces may exceed the limit of Constants.MULTIPART_UPLOAD_PART_NUM_LIMIT.

Parameters:

contentLength - the size of file.

minPartSize - the minimum size of multipart piece.

Returns:

a revisional size of multipart piece.

getCredentialsProvider

public static com.aliyun.oss.common.auth.CredentialsProvider getCredentialsProvider(URI uri,
                                                                                    org.apache.hadoop.conf.Configuration conf)
                                                                             throws IOException

Create credential provider specified by configuration, or create default credential provider if not specified.

Parameters:

uri - uri passed by caller

conf - configuration

Returns:

a credential provider

Throws:

IOException - on any problem. Class construction issues may be nested inside the IOE.

maybeAddTrailingSlash

public static String maybeAddTrailingSlash(String key)

Turns a path (relative or otherwise) into an OSS key, adding a trailing "/" if the path is not the root and does not already have a "/" at the end.

Parameters:

key - OSS key or ""

Returns:

the with a trailing "/", or, if it is the root key, "".

objectRepresentsDirectory

public static boolean objectRepresentsDirectory(String name,
                                                long size)

Check if OSS object represents a directory.

Parameters:

name - object key

size - object content length

Returns:

true if object represents a directory

createTmpFileForWrite

public static File createTmpFileForWrite(String path,
                                         long size,
                                         org.apache.hadoop.conf.Configuration conf)
                                  throws IOException

Demand create the directory allocator, then create a temporary file.

Parameters:

path - prefix for the temporary file

size - the size of the file that is going to be written

conf - the Configuration object

Returns:

a unique temporary file

Throws:

IOException - IO problems

getMultipartSizeProperty

public static long getMultipartSizeProperty(org.apache.hadoop.conf.Configuration conf,
                                            String property,
                                            long defVal)

Get a size property from the configuration: this property must be at least equal to Constants.MULTIPART_MIN_SIZE. If it is too small, it is rounded up to that minimum, and a warning printed.

Parameters:

conf - configuration

property - property name

defVal - default value

Returns:

the value, guaranteed to be above the minimum size