Package org.gorpipe.gor.manager

Class TableManager

java.lang.Object
org.gorpipe.gor.manager.TableManager
public class TableManager
extends java.lang.Object
Class to manage gor tables (dictionaries and tables).

Includes commands to:

load table insert entries into table delete entries from table select bucketize remove entries from buckets delete buckets

Created by gisli on 18/08/16.

TODO: Why not move bucketize methods into the DictionaryTable object. TODO: bucketzationInfo should be part of the table and stored in the table header (only passed in when table is created). Use sensible defaults otherwise.

  • Nested Class Summary

    Nested Classes 
    Modifier and Type Class Description
    static class  TableManager.Builder  

  • Field Summary

    Fields 
    Modifier and Type Field Description
    static java.time.Duration DEFAULT_LOCK_TIMEOUT  
    static java.lang.Class<? extends org.gorpipe.gor.table.lock.TableLock> DEFAULT_LOCK_TYPE  

  • Constructor Summary

    Constructors 
    Constructor Description
    TableManager()
    Default constructor.

  • Method Summary

    Modifier and Type Method Description
    void bucketize​(java.nio.file.Path tableFile, BucketManager.BucketPackLevel packLevel, int workers, int maxBucketCount, java.util.List<java.nio.file.Path> bucketDirs)
    Bucketize the given table.
    void delete​(java.nio.file.Path tableFile, org.gorpipe.gor.table.BaseTable.TableFilter entries)
    Delete entries from the table.
    void delete​(java.nio.file.Path tableFile, org.gorpipe.gor.table.BucketableTableEntry... entries)
    Delete entries from the table.
    void deleteBuckets​(java.nio.file.Path tableFile, java.nio.file.Path... buckets)
    Delete the given buckets.
    void deleteBuckets​(org.gorpipe.gor.table.BaseTable table, java.nio.file.Path... buckets)
    Delete the given buckets.
    int getBucketSize()  
    java.time.Duration getLockTimeout()  
    java.lang.Class<? extends org.gorpipe.gor.table.lock.TableLock> getLockType()  
    java.lang.String getSecurityContext()  
    org.gorpipe.gor.table.BaseTable initTable​(java.nio.file.Path path)
    Initialize the table given by path.
    void insert​(java.nio.file.Path tableFile, BucketManager.BucketPackLevel packLevel, int workers, org.gorpipe.gor.table.BucketableTableEntry... entries)
    Insert new data file (partition) into the table.
    void insert​(org.gorpipe.gor.table.BaseTable table, BucketManager.BucketPackLevel packLevel, int workers, org.gorpipe.gor.table.BucketableTableEntry... entries)
    Insert new data file (partition) into the table.
    static TableManager.Builder newBuilder()  
    void print​(org.gorpipe.gor.table.BaseTable.TableFilter lines)  
    void save​(org.gorpipe.gor.table.BaseTable table, BucketManager.BucketPackLevel packLevel, int workers)
    Save the given table.
    java.util.List<? extends org.gorpipe.gor.table.BucketableTableEntry> select​(java.nio.file.Path tableFile, java.lang.String[] files, java.lang.String[] aliases, java.lang.String[] tags, java.lang.String[] buckets, java.lang.String chrRange, boolean includedDeleted)
    Select entries from the table.
    java.util.Collection<? extends org.gorpipe.gor.table.BucketableTableEntry> selectAll​(java.nio.file.Path tableFile)
    Select all entries from table.
    void setBucketSize​(int bucketSize)
    Set the bucket size.
    void setMinBucketSize​(int minBucketSize)  
    void setSecurityContext​(java.lang.String securityContext)  

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • DEFAULT_LOCK_TIMEOUT

      public static final java.time.Duration DEFAULT_LOCK_TIMEOUT

    • DEFAULT_LOCK_TYPE

      public static final java.lang.Class<? extends org.gorpipe.gor.table.lock.TableLock> DEFAULT_LOCK_TYPE

  • Constructor Details

  • Method Details

    • newBuilder

      public static TableManager.Builder newBuilder()

    • setMinBucketSize

      public void setMinBucketSize​(int minBucketSize)

    • getBucketSize

      public int getBucketSize()

    • setBucketSize

      public void setBucketSize​(int bucketSize)
      Set the bucket size. Also updates the minBucketSize if it is larger than bucketSize.
      Parameters:
      bucketSize - the requested bucket size.

    • getSecurityContext

      public java.lang.String getSecurityContext()

    • setSecurityContext

      public void setSecurityContext​(java.lang.String securityContext)

    • getLockType

      public java.lang.Class<? extends org.gorpipe.gor.table.lock.TableLock> getLockType()

    • getLockTimeout

      public java.time.Duration getLockTimeout()

    • initTable

      public org.gorpipe.gor.table.BaseTable initTable​(java.nio.file.Path path)
      Initialize the table given by path.
      Parameters:
      path - path to the table.
      Returns:
      the table given by path.

    • insert

      public void insert​(java.nio.file.Path tableFile, BucketManager.BucketPackLevel packLevel, int workers, org.gorpipe.gor.table.BucketableTableEntry... entries)
      Insert new data file (partition) into the table. This might trigger bucketization.
      Parameters:
      tableFile - the path to the table file.
      packLevel - pack level to use (see BucketPackLevel).
      workers - number of workers to use for bucketization (if needed).
      entries - Files/lines to insert.

    • insert

      public void insert​(org.gorpipe.gor.table.BaseTable table, BucketManager.BucketPackLevel packLevel, int workers, org.gorpipe.gor.table.BucketableTableEntry... entries)
      Insert new data file (partition) into the table. This might trigger bucketization.
      Parameters:
      table - the table to insert into.
      packLevel - pack level to use (see BucketPackLevel).
      workers - number of workers to use for bucketization (if needed).
      entries - Files/lines to insert.

    • save

      public void save​(org.gorpipe.gor.table.BaseTable table, BucketManager.BucketPackLevel packLevel, int workers)
      Save the given table.
      Parameters:
      table - the table to save.
      packLevel - pack level to use (see BucketPackLevel).
      workers - number of workers to use for bucketization (if needed).

    • delete

      public void delete​(java.nio.file.Path tableFile, org.gorpipe.gor.table.BucketableTableEntry... entries)
      Delete entries from the table.
      Parameters:
      tableFile - path to the table file.
      entries - the entries to delete.

    • delete

      public void delete​(java.nio.file.Path tableFile, org.gorpipe.gor.table.BaseTable.TableFilter entries)
      Delete entries from the table.
      Parameters:
      tableFile - path to the table file.
      entries - the entries to delete.

    • select

      public java.util.List<? extends org.gorpipe.gor.table.BucketableTableEntry> select​(java.nio.file.Path tableFile, java.lang.String[] files, java.lang.String[] aliases, java.lang.String[] tags, java.lang.String[] buckets, java.lang.String chrRange, boolean includedDeleted)
      Select entries from the table.

      This function is useful to select entries as input into other methods of this class.

      Parameters:
      tableFile - path to the table file.
      files - list of files to include.
      aliases - list of aliases to include.
      tags - list of tags to include.
      buckets - list of buckets to include.
      chrRange - filter range, string in format [chr from]:[pos from]-[chr to]:[pos to]. Example: chr1:10000-chr1:200000
      includedDeleted - Should deleted files be included in the result.
      Returns:
      entries from the table, matching the given criteria.

    • selectAll

      public java.util.Collection<? extends org.gorpipe.gor.table.BucketableTableEntry> selectAll​(java.nio.file.Path tableFile)
      Select all entries from table.

      This function is useful to select entries as input into other methods of this class.

      Parameters:
      tableFile - path to the table file.
      Returns:
      all entries from table as a collection.

    • print

      public void print​(org.gorpipe.gor.table.BaseTable.TableFilter lines)

    • bucketize

      public void bucketize​(java.nio.file.Path tableFile, BucketManager.BucketPackLevel packLevel, int workers, int maxBucketCount, java.util.List<java.nio.file.Path> bucketDirs)
      Bucketize the given table.
      Parameters:
      tableFile - the path to the table file.
      packLevel - pack level to use (see BucketPackLevel).
      workers - number of workers to use for bucketization (if needed).
      maxBucketCount - Maximum number of buckets to generate on this call.
      bucketDirs - array of directories to bucketize to, ignored if null. The dirs are absolute or relative to the table dir.

    • deleteBuckets

      public void deleteBuckets​(org.gorpipe.gor.table.BaseTable table, java.nio.file.Path... buckets)
      Delete the given buckets.
      Parameters:
      buckets - list of buckets to be deleted.

    • deleteBuckets

      public void deleteBuckets​(java.nio.file.Path tableFile, java.nio.file.Path... buckets)
      Delete the given buckets.
      Parameters:
      tableFile - the path to the table file.
      buckets - list of buckets to be deleted.