org.calrissian.accumulorecipes.eventstore.impl

Class AccumuloEventStore

java.lang.Object

org.calrissian.accumulorecipes.eventstore.impl.AccumuloEventStore

All Implemented Interfaces:

EventStore

public class AccumuloEventStore
extends Object
implements EventStore

The Accumulo implementation of the EventStore which uses deterministic sharding to distribute ingest/queries over the cloud to speed them up.

Field Summary

Fields

Modifier and Type	Field and Description
static String	DEFAULT_IDX_TABLE_NAME
static EventShardBuilder	DEFAULT_SHARD_BUILDER
static String	DEFAULT_SHARD_TABLE_NAME
static StoreConfig	DEFAULT_STORE_CONFIG

Constructor Summary

Constructors

Constructor and Description
AccumuloEventStore(org.apache.accumulo.core.client.Connector connector)
AccumuloEventStore(org.apache.accumulo.core.client.Connector connector, String indexTable, String shardTable, StoreConfig config, org.calrissian.mango.types.TypeRegistry<String> typeRegistry)

Method Summary

Methods

Modifier and Type	Method and Description
org.calrissian.mango.collect.CloseableIterable<org.calrissian.mango.domain.event.Event>	get(Collection<org.calrissian.mango.domain.event.EventIndex> uuids, Set<String> selectFields, Auths auths) This method will batch get a bunch of events by uuid (and optionally timestamp).
org.calrissian.mango.collect.CloseableIterable<org.calrissian.mango.domain.event.Event>	query(Date start, Date end, org.calrissian.mango.criteria.domain.Node query, Set<String> selectFields, Auths auths) Shard ids for which to scan are generated from the given start and end time.
void	save(Iterable<? extends org.calrissian.mango.domain.event.Event> events) Events get save into a sharded table to parallelize queries & ingest.
void	shutdown() Free up any resources used by the store.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_IDX_TABLE_NAME

public static final String DEFAULT_IDX_TABLE_NAME

See Also:

Constant Field Values

DEFAULT_SHARD_TABLE_NAME

public static final String DEFAULT_SHARD_TABLE_NAME

See Also:

Constant Field Values

DEFAULT_STORE_CONFIG

public static final StoreConfig DEFAULT_STORE_CONFIG

DEFAULT_SHARD_BUILDER

public static final EventShardBuilder DEFAULT_SHARD_BUILDER

Constructor Detail

AccumuloEventStore

public AccumuloEventStore(org.apache.accumulo.core.client.Connector connector)
                   throws org.apache.accumulo.core.client.TableExistsException,
                          org.apache.accumulo.core.client.AccumuloSecurityException,
                          org.apache.accumulo.core.client.AccumuloException,
                          org.apache.accumulo.core.client.TableNotFoundException

Throws:

org.apache.accumulo.core.client.TableExistsException

org.apache.accumulo.core.client.AccumuloSecurityException

org.apache.accumulo.core.client.AccumuloException

org.apache.accumulo.core.client.TableNotFoundException

AccumuloEventStore

public AccumuloEventStore(org.apache.accumulo.core.client.Connector connector,
                  String indexTable,
                  String shardTable,
                  StoreConfig config,
                  org.calrissian.mango.types.TypeRegistry<String> typeRegistry)
                   throws org.apache.accumulo.core.client.TableExistsException,
                          org.apache.accumulo.core.client.AccumuloSecurityException,
                          org.apache.accumulo.core.client.AccumuloException,
                          org.apache.accumulo.core.client.TableNotFoundException

Throws:

org.apache.accumulo.core.client.TableExistsException

org.apache.accumulo.core.client.AccumuloSecurityException

org.apache.accumulo.core.client.AccumuloException

org.apache.accumulo.core.client.TableNotFoundException

Method Detail

shutdown

public void shutdown()
              throws org.apache.accumulo.core.client.MutationsRejectedException

Free up any resources used by the store.

Throws:

org.apache.accumulo.core.client.MutationsRejectedException

save

public void save(Iterable<? extends org.calrissian.mango.domain.event.Event> events)

Events get save into a sharded table to parallelize queries & ingest. Since the data is temporal by default, an index table allows the lookup of events by UUID only (when the event's timestamp is not known).

Specified by:

save in interface EventStore

Parameters:

events -

Throws:

Exception

query

public org.calrissian.mango.collect.CloseableIterable<org.calrissian.mango.domain.event.Event> query(Date start,
                                                                                            Date end,
                                                                                            org.calrissian.mango.criteria.domain.Node query,
                                                                                            Set<String> selectFields,
                                                                                            Auths auths)

Shard ids for which to scan are generated from the given start and end time. The given query specifies which events to return. It is propagated all the way down to the iterators so that the query is run in parallel on each tablet that needs to be scanned.

Specified by:

query in interface EventStore

Returns:

get

public org.calrissian.mango.collect.CloseableIterable<org.calrissian.mango.domain.event.Event> get(Collection<org.calrissian.mango.domain.event.EventIndex> uuids,
                                                                                          Set<String> selectFields,
                                                                                          Auths auths)

This method will batch get a bunch of events by uuid (and optionally timestamp). If another store is used to index into events in this store in a specially designed way (i.e. getting the last-n events, etc...) then the most optimal way to store the index would be the UUID and the timestamp. However, if all that is known about an event is the uuid, this method will do the extra fetch of the timestamp from the index table.

Specified by:

get in interface EventStore

Returns: