Package org.gorpipe.gor.manager

Class BucketManager<T extends org.gorpipe.gor.table.BucketableTableEntry>

java.lang.Object
org.gorpipe.gor.manager.BucketManager<T>
public class BucketManager<T extends org.gorpipe.gor.table.BucketableTableEntry>
extends java.lang.Object

  • Field Details

  • Constructor Details

    • BucketManager

      public BucketManager​(org.gorpipe.gor.table.BaseTable<T> table)
      Default constructor.

  • Method Details

    • newBuilder

      public static BucketManager.Builder newBuilder​(org.gorpipe.gor.table.BaseTable table)

    • bucketize

      public void bucketize()

    • bucketize

      public int bucketize​(BucketManager.BucketPackLevel packLevel, int maxBucketCount)

    • bucketize

      public int bucketize​(BucketManager.BucketPackLevel packLevel, int maxBucketCount, java.util.List<java.nio.file.Path> bucketDirs, boolean forceClean)
      Bucketize the given table.
      Parameters:
      packLevel - pack level to use (see BucketPackLevel).
      maxBucketCount - Maximum number of buckets to generate on this call.
      bucketDirs - array of directories to bucketize to, ignored if null. The dirs are absolute or relative to the table dir.
      forceClean - Should we force clean bucket files (if force clean we ignoring grace periods).
      Returns:
      buckets created.

    • deleteBuckets

      public void deleteBuckets​(java.nio.file.Path... buckets)
      Delete the given bucket.
      Parameters:
      buckets - list of buckets to be deleted.

    • getEffectiveMinBucketSize

      protected int getEffectiveMinBucketSize()
      Get effective min bucket size.
      Returns:
      effective min bucket size.

    • getBucketSize

      public int getBucketSize()

    • setBucketSize

      public void setBucketSize​(int bucketSize)

    • getMinBucketSize

      public int getMinBucketSize()

    • setMinBucketSize

      public void setMinBucketSize​(int minBucketSize)

    • getDefaultBucketDir

      protected java.nio.file.Path getDefaultBucketDir()

    • getLockTimeout

      public java.time.Duration getLockTimeout()

    • setBucketDirs

      public void setBucketDirs​(java.util.List<java.nio.file.Path> newBucketDirs)
      Set the bucket dir list.
      Parameters:
      newBucketDirs - the new list of bucket dirs, absolute or relative to table dir.

    • pickBucketDir

      protected final java.nio.file.Path pickBucketDir()
      Pick a bucket dir from a list. There couple of strategies to choose from (using the gor.table.buckets.directory.strategy option.) 'random': pick a directory by random. 'least_used': pick the directory that is least used by the dictionary. Default.
      Returns:
      path to the chosen bucket dir, relative to path or absolute.

    • cleanBucketFiles

      protected void cleanBucketFiles​(org.gorpipe.gor.table.lock.TableLock bucketizeLock, boolean force) throws java.io.IOException
      Cleans up bucketFiles that are not in use and have not been accessed for a given period of time.

      TODO: We are not handling changed bucket dirs very well, i.e. if we remove bucket dir from the list of bucket dirs because we don't want add more files to that directory it will not be included in the cleaning. Hard to changes this until we add the bucket dirs to the table header (then we can keep two lists). Note, not enough to check the directories in use in the file as we remove them from the gord file before actual delete the files (because of the delayed delete).

      Parameters:
      bucketizeLock - the bucketize lock to use.
      force - force cleaning, ignoring grace periods.
      Throws:
      java.io.IOException