org.calrissian.accumulorecipes.featurestore.impl

Class AccumuloFeatureStore

java.lang.Object

org.calrissian.accumulorecipes.featurestore.impl.AccumuloFeatureStore

All Implemented Interfaces:

FeatureStore

public class AccumuloFeatureStore
extends Object
implements FeatureStore

This class will store simple feature vectors into accumulo. The features will aggregate over predefined time intervals and the aggregations are available immediately.

Format of the table: Rowid CF CQ Value grouprevTS model'MINUTES' typename value grouprevTS model'HOURS' typename value grouprevTS model'DAYS' typename value grouprevTS model'MONTHS' typename value

Field Summary

Fields

Modifier and Type	Field and Description
protected static StoreConfig	DEFAULT_STORE_CONFIG
static String	DEFAULT_TABLE_NAME
protected FeatureRegistry	registry
static String	REVERSE_SUFFIX

Constructor Summary

Constructors

Constructor and Description
AccumuloFeatureStore(org.apache.accumulo.core.client.Connector connector)
AccumuloFeatureStore(org.apache.accumulo.core.client.Connector connector, String tableName, StoreConfig config, FeatureRegistry featureRegistry)

Method Summary

Methods

Modifier and Type	Method and Description
protected <T extends Feature> FeatureTransform<T>	buildFeatureTransform(AccumuloFeatureConfig xform)
protected org.apache.accumulo.core.data.Range	buildRange(String type, Date start, Date end, TimeUnit timeUnit)
void	flush()
void	initialize()
protected org.apache.accumulo.core.client.ScannerBase	metricScanner(AccumuloFeatureConfig xform, Date start, Date end, String group, Iterable<String> types, String name, TimeUnit timeUnit, Auths auths)
protected org.apache.accumulo.core.client.ScannerBase	metricScanner(AccumuloFeatureConfig xform, Date start, Date end, String group, String type, String name, TimeUnit timeUnit, Auths auths)
<T extends Feature> org.calrissian.mango.collect.CloseableIterable<T>	query(Date start, Date end, String group, Set<String> types, String name, TimeUnit timeUnit, Class<T> featureType, Auths auths) Queries features back from the store for a group and multiple types
<T extends Feature> org.calrissian.mango.collect.CloseableIterable<T>	query(Date start, Date end, String group, String type, String name, TimeUnit timeUnit, Class<T> featureType, Auths auths) Query features back from the store.
void	save(Iterable<? extends Feature> featureData) Save a set of features to the store
void	save(Iterable<? extends Feature> featureData, Iterable<TimeUnit> timeUnits) Save a set of features to the store for the specified time units
void	shutdown() Will close all underlying resources

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_TABLE_NAME

public static final String DEFAULT_TABLE_NAME

See Also:

Constant Field Values

REVERSE_SUFFIX

public static final String REVERSE_SUFFIX

See Also:

Constant Field Values

DEFAULT_STORE_CONFIG

protected static final StoreConfig DEFAULT_STORE_CONFIG

registry

protected final FeatureRegistry registry

Constructor Detail

AccumuloFeatureStore

public AccumuloFeatureStore(org.apache.accumulo.core.client.Connector connector)
                     throws org.apache.accumulo.core.client.TableNotFoundException,
                            org.apache.accumulo.core.client.TableExistsException,
                            org.apache.accumulo.core.client.AccumuloSecurityException,
                            org.apache.accumulo.core.client.AccumuloException

Throws:

org.apache.accumulo.core.client.TableNotFoundException

org.apache.accumulo.core.client.TableExistsException

org.apache.accumulo.core.client.AccumuloSecurityException

org.apache.accumulo.core.client.AccumuloException

AccumuloFeatureStore

public AccumuloFeatureStore(org.apache.accumulo.core.client.Connector connector,
                    String tableName,
                    StoreConfig config,
                    FeatureRegistry featureRegistry)
                     throws org.apache.accumulo.core.client.TableNotFoundException,
                            org.apache.accumulo.core.client.TableExistsException,
                            org.apache.accumulo.core.client.AccumuloSecurityException,
                            org.apache.accumulo.core.client.AccumuloException

Throws:

org.apache.accumulo.core.client.TableNotFoundException

org.apache.accumulo.core.client.TableExistsException

org.apache.accumulo.core.client.AccumuloSecurityException

org.apache.accumulo.core.client.AccumuloException

Method Detail

initialize

public void initialize()
                throws org.apache.accumulo.core.client.AccumuloException,
                       org.apache.accumulo.core.client.AccumuloSecurityException,
                       org.apache.accumulo.core.client.TableNotFoundException,
                       org.apache.accumulo.core.client.TableExistsException

Throws:

org.apache.accumulo.core.client.AccumuloException

org.apache.accumulo.core.client.AccumuloSecurityException

org.apache.accumulo.core.client.TableNotFoundException

org.apache.accumulo.core.client.TableExistsException

metricScanner

protected org.apache.accumulo.core.client.ScannerBase metricScanner(AccumuloFeatureConfig xform,
                                                        Date start,
                                                        Date end,
                                                        String group,
                                                        Iterable<String> types,
                                                        String name,
                                                        TimeUnit timeUnit,
                                                        Auths auths)

metricScanner

protected org.apache.accumulo.core.client.ScannerBase metricScanner(AccumuloFeatureConfig xform,
                                                        Date start,
                                                        Date end,
                                                        String group,
                                                        String type,
                                                        String name,
                                                        TimeUnit timeUnit,
                                                        Auths auths)

buildRange

protected org.apache.accumulo.core.data.Range buildRange(String type,
                                             Date start,
                                             Date end,
                                             TimeUnit timeUnit)

shutdown

public void shutdown()
              throws org.apache.accumulo.core.client.MutationsRejectedException

Will close all underlying resources

Throws:

org.apache.accumulo.core.client.MutationsRejectedException

save

public void save(Iterable<? extends Feature> featureData)

Description copied from interface: FeatureStore

Save a set of features to the store

Specified by:

save in interface FeatureStore

save

public void save(Iterable<? extends Feature> featureData,
        Iterable<TimeUnit> timeUnits)

Save a set of features to the store for the specified time units

Specified by:

save in interface FeatureStore

flush

public void flush()
           throws Exception

Specified by:

flush in interface FeatureStore

Throws:

Exception

query

public <T extends Feature> org.calrissian.mango.collect.CloseableIterable<T> query(Date start,
                                                                          Date end,
                                                                          String group,
                                                                          String type,
                                                                          String name,
                                                                          TimeUnit timeUnit,
                                                                          Class<T> featureType,
                                                                          Auths auths)

Query features back from the store.

Specified by:

query in interface FeatureStore

query

public <T extends Feature> org.calrissian.mango.collect.CloseableIterable<T> query(Date start,
                                                                          Date end,
                                                                          String group,
                                                                          Set<String> types,
                                                                          String name,
                                                                          TimeUnit timeUnit,
                                                                          Class<T> featureType,
                                                                          Auths auths)

Description copied from interface: FeatureStore

Queries features back from the store for a group and multiple types

Specified by:

query in interface FeatureStore

buildFeatureTransform

protected <T extends Feature> FeatureTransform<T> buildFeatureTransform(AccumuloFeatureConfig xform)