Package org.gorpipe.gor.manager

Class TableManager

java.lang.Object

org.gorpipe.gor.manager.TableManager

public class TableManager
extends java.lang.Object

Class to manage gor tables (dictionaries and tables).

Includes commands to:

load table insert entries into table delete entries from table select bucketize remove entries from buckets delete buckets

Created by gisli on 18/08/16.

TODO: Why not move bucketize methods into the DictionaryTable object. TODO: bucketzationInfo should be part of the table and stored in the table header (only passed in when table is created). Use sensible defaults otherwise.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	TableManager.Builder

Field Summary

Fields

Modifier and Type	Field	Description
static java.time.Duration	DEFAULT_LOCK_TIMEOUT
static java.lang.Class<? extends org.gorpipe.gor.table.lock.TableLock>	DEFAULT_LOCK_TYPE

Constructor Summary

Constructors

Constructor	Description
TableManager()	Default constructor.

Method Summary

Modifier and Type	Method	Description
void	bucketize(java.nio.file.Path tableFile, BucketManager.BucketPackLevel packLevel, int workers, int maxBucketCount, java.util.List<java.nio.file.Path> bucketDirs)	Bucketize the given table.
void	delete(java.nio.file.Path tableFile, org.gorpipe.gor.table.dictionary.BaseDictionaryTable.TableFilter entries)	Delete entries from the table.
void	delete(java.nio.file.Path tableFile, org.gorpipe.gor.table.dictionary.BucketableTableEntry... entries)	Delete entries from the table.
void	deleteBuckets(java.nio.file.Path tableFile, java.nio.file.Path... buckets)	Delete the given buckets.
void	deleteBuckets(org.gorpipe.gor.table.dictionary.BaseDictionaryTable table, java.nio.file.Path... buckets)	Delete the given buckets.
int	getBucketSize()
java.time.Duration	getLockTimeout()
java.lang.Class<? extends org.gorpipe.gor.table.lock.TableLock>	getLockType()
java.lang.String	getSecurityContext()
org.gorpipe.gor.table.dictionary.BaseDictionaryTable	initTable(java.nio.file.Path path)	Initialize the table given by path.
void	insert(java.nio.file.Path tableFile, BucketManager.BucketPackLevel packLevel, int workers, org.gorpipe.gor.table.dictionary.BucketableTableEntry... entries)	Insert new data file (partition) into the table.
void	insert(org.gorpipe.gor.table.dictionary.BaseDictionaryTable table, BucketManager.BucketPackLevel packLevel, int workers, org.gorpipe.gor.table.dictionary.BucketableTableEntry... entries)	Insert new data file (partition) into the table.
static TableManager.Builder	newBuilder()
void	print(org.gorpipe.gor.table.dictionary.BaseDictionaryTable.TableFilter lines)
void	save(org.gorpipe.gor.table.dictionary.BaseDictionaryTable table, BucketManager.BucketPackLevel packLevel, int workers)	Save the given table.
java.util.List<? extends org.gorpipe.gor.table.dictionary.BucketableTableEntry>	select(java.nio.file.Path tableFile, java.lang.String[] files, java.lang.String[] aliases, java.lang.String[] tags, java.lang.String[] buckets, java.lang.String chrRange, boolean includedDeleted)	Select entries from the table.
java.util.Collection<? extends org.gorpipe.gor.table.dictionary.BucketableTableEntry>	selectAll(java.nio.file.Path tableFile)	Select all entries from table.
void	setBucketSize(int bucketSize)	Set the bucket size.
void	setMinBucketSize(int minBucketSize)
void	setSecurityContext(java.lang.String securityContext)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_LOCK_TIMEOUT

public static final java.time.Duration DEFAULT_LOCK_TIMEOUT

DEFAULT_LOCK_TYPE

public static final java.lang.Class<? extends org.gorpipe.gor.table.lock.TableLock> DEFAULT_LOCK_TYPE

Constructor Detail

TableManager

public TableManager()

Default constructor.

Method Detail

newBuilder

public static TableManager.Builder newBuilder()

setMinBucketSize

public void setMinBucketSize(int minBucketSize)

getBucketSize

public int getBucketSize()

setBucketSize

public void setBucketSize(int bucketSize)

Set the bucket size. Also updates the minBucketSize if it is larger than bucketSize.

Parameters:

bucketSize - the requested bucket size.

getSecurityContext

public java.lang.String getSecurityContext()

setSecurityContext

public void setSecurityContext(java.lang.String securityContext)

getLockType

public java.lang.Class<? extends org.gorpipe.gor.table.lock.TableLock> getLockType()

getLockTimeout

public java.time.Duration getLockTimeout()

initTable

public org.gorpipe.gor.table.dictionary.BaseDictionaryTable initTable(java.nio.file.Path path)

Initialize the table given by path.

Parameters:

path - path to the table.

Returns:

the table given by path.

insert

public void insert(java.nio.file.Path tableFile,
                   BucketManager.BucketPackLevel packLevel,
                   int workers,
                   org.gorpipe.gor.table.dictionary.BucketableTableEntry... entries)

Insert new data file (partition) into the table. This might trigger bucketization.

Parameters:

tableFile - the path to the table file.

packLevel - pack level to use (see BucketPackLevel).

workers - number of workers to use for bucketization (if needed).

entries - Files/lines to insert.

insert

public void insert(org.gorpipe.gor.table.dictionary.BaseDictionaryTable table,
                   BucketManager.BucketPackLevel packLevel,
                   int workers,
                   org.gorpipe.gor.table.dictionary.BucketableTableEntry... entries)

Insert new data file (partition) into the table. This might trigger bucketization.

Parameters:

table - the table to insert into.

packLevel - pack level to use (see BucketPackLevel).

workers - number of workers to use for bucketization (if needed).

entries - Files/lines to insert.

save

public void save(org.gorpipe.gor.table.dictionary.BaseDictionaryTable table,
                 BucketManager.BucketPackLevel packLevel,
                 int workers)

Save the given table.

Parameters:

table - the table to save.

packLevel - pack level to use (see BucketPackLevel).

workers - number of workers to use for bucketization (if needed).

delete

public void delete(java.nio.file.Path tableFile,
                   org.gorpipe.gor.table.dictionary.BucketableTableEntry... entries)

Delete entries from the table.

Parameters:

tableFile - path to the table file.

entries - the entries to delete.

delete

public void delete(java.nio.file.Path tableFile,
                   org.gorpipe.gor.table.dictionary.BaseDictionaryTable.TableFilter entries)

Delete entries from the table.

Parameters:

tableFile - path to the table file.

entries - the entries to delete.

select

public java.util.List<? extends org.gorpipe.gor.table.dictionary.BucketableTableEntry> select(java.nio.file.Path tableFile,
                                                                                              java.lang.String[] files,
                                                                                              java.lang.String[] aliases,
                                                                                              java.lang.String[] tags,
                                                                                              java.lang.String[] buckets,
                                                                                              java.lang.String chrRange,
                                                                                              boolean includedDeleted)

Select entries from the table.

This function is useful to select entries as input into other methods of this class.

Parameters:

tableFile - path to the table file.

files - list of files to include.

aliases - list of aliases to include.

tags - list of tags to include.

buckets - list of buckets to include.

chrRange - filter range, string in format [chr from]:[pos from]-[chr to]:[pos to]. Example: chr1:10000-chr1:200000

includedDeleted - Should deleted files be included in the result.

Returns:

entries from the table, matching the given criteria.

selectAll

public java.util.Collection<? extends org.gorpipe.gor.table.dictionary.BucketableTableEntry> selectAll(java.nio.file.Path tableFile)

Select all entries from table.

This function is useful to select entries as input into other methods of this class.

Parameters:

tableFile - path to the table file.

Returns:

all entries from table as a collection.

print

public void print(org.gorpipe.gor.table.dictionary.BaseDictionaryTable.TableFilter lines)

bucketize

public void bucketize(java.nio.file.Path tableFile,
                      BucketManager.BucketPackLevel packLevel,
                      int workers,
                      int maxBucketCount,
                      java.util.List<java.nio.file.Path> bucketDirs)

Bucketize the given table.

Parameters:

tableFile - the path to the table file.

packLevel - pack level to use (see BucketPackLevel).

workers - number of workers to use for bucketization (if needed).

maxBucketCount - Maximum number of buckets to generate on this call.

bucketDirs - array of directories to bucketize to, ignored if null. The dirs are absolute or relative to the table dir.

deleteBuckets

public void deleteBuckets(org.gorpipe.gor.table.dictionary.BaseDictionaryTable table,
                          java.nio.file.Path... buckets)

Delete the given buckets.

Parameters:

buckets - list of buckets to be deleted.

deleteBuckets

public void deleteBuckets(java.nio.file.Path tableFile,
                          java.nio.file.Path... buckets)

Delete the given buckets.

Parameters:

tableFile - the path to the table file.

buckets - list of buckets to be deleted.