Package org.gorpipe.gor.manager

Class BucketManager<T extends org.gorpipe.gor.table.BucketableTableEntry>

  • public class BucketManager<T extends org.gorpipe.gor.table.BucketableTableEntry>
extends java.lang.Object

    • Field Detail

      • DEFAULT_MIN_BUCKET_SIZE

        public static final int DEFAULT_MIN_BUCKET_SIZE
        See Also:
        Constant Field Values

      • DEFAULT_MAX_BUCKET_COUNT

        public static final int DEFAULT_MAX_BUCKET_COUNT
        See Also:
        Constant Field Values

      • DEFAULT_LOCK_TIMEOUT

        public static final java.time.Duration DEFAULT_LOCK_TIMEOUT

      • DEFAULT_LOCK_TYPE

        public static final java.lang.Class<? extends org.gorpipe.gor.table.lock.TableLock> DEFAULT_LOCK_TYPE

      • HEADER_MIN_BUCKET_SIZE_KEY

        public static final java.lang.String HEADER_MIN_BUCKET_SIZE_KEY
        See Also:
        Constant Field Values

      • HEADER_BUCKET_SIZE_KEY

        public static final java.lang.String HEADER_BUCKET_SIZE_KEY
        See Also:
        Constant Field Values

      • HEADER_BUCKET_DIRS_KEY

        public static final java.lang.String HEADER_BUCKET_DIRS_KEY
        See Also:
        Constant Field Values

      • HEADER_BUCKET_MAX_BUCKETS

        public static final java.lang.String HEADER_BUCKET_MAX_BUCKETS
        See Also:
        Constant Field Values

      • gracePeriodForDeletingBuckets

        protected java.time.Duration gracePeriodForDeletingBuckets

    • Constructor Detail

      • BucketManager

        public BucketManager​(org.gorpipe.gor.table.BaseTable<T> table)
        Default constructor.

    • Method Detail

      • bucketize

        public void bucketize()

      • bucketize

        public int bucketize​(BucketManager.BucketPackLevel packLevel,
                     int maxBucketCount,
                     java.util.List<java.nio.file.Path> bucketDirs,
                     boolean forceClean)
        Bucketize the given table.
        Parameters:
        packLevel - pack level to use (see BucketPackLevel).
        maxBucketCount - Maximum number of buckets to generate on this call, 0 or less means not set in which case we read it from the table, if its not set on the table we use default.
        bucketDirs - array of directories to bucketize to, ignored if null. The dirs are absolute or relative to the table dir.
        forceClean - Should we force clean bucket files (if force clean we ignoring grace periods).
        Returns:
        buckets created.

      • deleteBuckets

        public void deleteBuckets​(java.nio.file.Path... buckets)
        Delete the given buckets.
        Parameters:
        buckets - list of buckets to be deleted.

      • deleteBuckets

        public void deleteBuckets​(boolean force,
                          java.nio.file.Path... buckets)
        Delete the given buckets.
        Parameters:
        force - should grace period be ignored.
        buckets - list of buckets to be deleted.

      • getEffectiveMinBucketSize

        protected int getEffectiveMinBucketSize()
        Get effective min bucket size.
        Returns:
        effective min bucket size.

      • getBucketSize

        public int getBucketSize()

      • setBucketSize

        public void setBucketSize​(int bucketSize)

      • getMinBucketSize

        public int getMinBucketSize()

      • setMinBucketSize

        public void setMinBucketSize​(int minBucketSize)

      • getDefaultBucketDir

        protected java.nio.file.Path getDefaultBucketDir()

      • getLockTimeout

        public java.time.Duration getLockTimeout()

      • setBucketDirs

        public void setBucketDirs​(java.util.List<java.nio.file.Path> newBucketDirs)
        Set the bucket dir list.
        Parameters:
        newBucketDirs - the new list of bucket dirs, absolute or relative to table dir.

      • pickBucketDir

        protected final java.nio.file.Path pickBucketDir()
        Pick a bucket dir from a list. There couple of strategies to choose from (using the gor.table.buckets.directory.strategy option.) 'random': pick a directory by random. 'least_used': pick the directory that is least used by the dictionary. Default.
        Returns:
        path to the chosen bucket dir, relative to path or absolute.

      • cleanOldBucketFiles

        protected void cleanOldBucketFiles​(org.gorpipe.gor.table.lock.TableLock bucketizeLock,
                                   boolean force)
                            throws java.io.IOException
        Cleans up bucketFiles that are not in use and have not been accessed for a given period of time.

        Notes: 1. We only manage bucketdirs that are listed in the table meta data and are in use. So if we stop using a folder and there are no buckets there listed in the table then we need to manage it manually.

        Parameters:
        bucketizeLock - the bucketize lock to use.
        force - force cleaning, ignoring grace periods.
        Throws:
        java.io.IOException