Partitioner

A method for Mothra packers to determine into which single partition (if any) on one level of a partitioning hierarchy a Record should be stored. Partitioners are used to partition (split) records into various directories in the long-term data storage location.

A single Partitioner typically splits on one attribute of a Record, such as its source IP address, IP protocol, or destination port.

A sequence of multiple Partitioners is used by PartitionerConfigurator to partition on multiple attributes in order.

See also: the companion object for numerous ways to define simple Partitioners on various data types. These allow you to quickly specify ranges or specific values for different partitions.

Linear Supertypes

AnyRef, Any

Known Subclasses

Float32, Float64, Int16, Int32, Int64, Int8, NumericPartitioner, UInt16, UInt32, UInt64, UInt8, UniqueValue

Abstract Value Members

abstract def pathForFilename(filename: String): Option[String]

Returns the path component extracted from a String representing a filename created by super_mediator running in invariant mode as an Option.
Returns the path component extracted from a String representing a filename created by super_mediator running in invariant mode as an Option. Returns None when either the String does not include the field or when the Partitioner does not designate a specific path component.
abstract def pathForRecord(rec: Record): Option[String]

Returns the path component for the Record rec based on this Partitioner as an Option.
Returns the path component for the Record rec based on this Partitioner as an Option. Returns None when either the Record lacks the necessary information for the Partitioner or when the Partitioner does not designate a specific path component for the Record.

Concrete Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
val delim: String

In the directory name generated by this partitoiner, the delimiter between the operator, field name, and argument(s)
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
def hashCode(): Int

Definition Classes
AnyRef → Any
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
val max_path_component: Int

The maximum length of a path component that the file system supports.
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
AnyRef → Any
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

Related Docs: object Partitioner | package packer

trait Partitioner extends AnyRef

Abstract Value Members

abstract def pathForFilename(filename: String): Option[String]

abstract def pathForRecord(rec: Record): Option[String]

Concrete Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

final def asInstanceOf[T0]: T0

def clone(): AnyRef

val delim: String

final def eq(arg0: AnyRef): Boolean

def equals(arg0: Any): Boolean

def finalize(): Unit

final def getClass(): Class[_]

def hashCode(): Int

final def isInstanceOf[T0]: Boolean

val max_path_component: Int

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

final def synchronized[T0](arg0: ⇒ T0): T0

def toString(): String

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

Inherited from AnyRef

Inherited from Any

Ungrouped