Packages

  • package root
    Definition Classes
    root
  • package org
    Definition Classes
    root
  • package cert
    Definition Classes
    org
  • package netsa
    Definition Classes
    cert
  • package mothra
    Definition Classes
    netsa
  • package packer

    The public traits and classes in this package are used to provide configuration for the Mothra packing tools.

    The public traits and classes in this package are used to provide configuration for the Mothra packing tools. (mothra-packer, mothra-invariantpacker, and mothra-repacker.)

    All three tools may be configured using PartitionerConfigurator objects, which provide a sequence of Partitioners to determine which single partition directory within a Mothra repository should receive each record. This is the recommended method for configuring partitioning.

    For more specialized needs, outside the standard Mothra repository structure, PackingLogic objects may produce any number of output PackableRecords for each input Record, allowing for arbitrary directory structures, record re-writing, record filtering, and other possibilities. We do not recommend using this configuration method unless you have very specific needs.

    Definition Classes
    mothra
  • object Partitioner

    An object to hold classes that extend the [Partitioner] trait.

    An object to hold classes that extend the [Partitioner] trait.

    Definition Classes
    packer
  • Float32
  • Float64
  • Int16
  • Int32
  • Int64
  • Int8
  • NumericPartitioner
  • UInt16
  • UInt32
  • UInt64
  • UInt8
  • UniqueValue
c

org.cert.netsa.mothra.packer.Partitioner

NumericPartitioner

sealed abstract class NumericPartitioner[T] extends Partitioner with StrictLogging

NumericPartitioner supports partitioning based on the numeric field named key in a Record, where each partition represents records where that field either has a specific (single) value or falls with a range of values.

NumericPartitioner is the parent class of other classes for specific numeric types (such as 16-bit integers and 32-bit floating point numbers). These subclasses must specify a numeric type and the allowable range of values for that type.

T

The type of the number

Linear Supertypes
StrictLogging, Partitioner, AnyRef, Any
Known Subclasses
Float32, Float64, Int16, Int32, Int64, Int8, UInt16, UInt32, UInt64, UInt8
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. NumericPartitioner
  2. StrictLogging
  3. Partitioner
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. def addMultiplePartitionsEqualsAny(s: Seq[T]): Unit

    Add multiple partitions for when Record value x equals any value in the list.

    Add multiple partitions for when Record value x equals any value in the list. This is a convenience function that invokes addPartitionEquals for each value in s.

    Exceptions thrown

    java.lang.IllegalArgumentException when an existing interval includes any value in s or when v is outside the range min–max.

    See also

    addSinglePartitionEqualsAny for a method that puts multiple values in a single partition

  5. def addNotIn(): Unit

    Adds a single additional rule to this Partitioner that covers all values that are not currently covered by an existing partition.

    Adds a single additional rule to this Partitioner that covers all values that are not currently covered by an existing partition. This method may only be used when the Partitioner does not contain any ranges; that is, when only addPartitionEquals, addMultiplePartitionsEqualsAny, and addSinglePartitionEqualsAny have been used.

    Exceptions thrown

    java.lang.RuntimeException when the resulting path component length would be larger than max_path_component, when the Partitioner contains ranges, or when the Partitioner is empty.

    See also

    coverRanges for an alternative way to handle remaining values

  6. def addPartitionEquals(v: T): Unit

    Add a partition for when Record value x == v.

    Add a partition for when Record value x == v.

    Exceptions thrown

    java.lang.IllegalArgumentException when an existing interval includes v or when v is outside the range min–max.

  7. def addPartitionGreaterEquals(v: T): Unit

    Add a partition for when Record value x >= v.

    Add a partition for when Record value x >= v. Since partitions may not overlap, this method and addPartitionGreaterThan may only be called one time for a Partitioner.

    Exceptions thrown

    java.lang.IllegalArgumentException when an existing interval includes any value greater than or equal to v or when v is outside the range min–max.

  8. def addPartitionGreaterThan(v: T): Unit

    Add a partition for when Record value x > v.

    Add a partition for when Record value x > v. Since partitions may not overlap, this method and addPartitionGreaterEquals may only be called one time for a Partitioner.

    Exceptions thrown

    java.lang.IllegalArgumentException when an existing interval includes any value greater than v, when v is outside the range min–max, or when v equals max.

  9. def addPartitionLessEquals(v: T): Unit

    Add a partition for when Record value x <= v.

    Add a partition for when Record value x <= v. Since partitions may not overlap, this method and addPartitionLessThan may only be called one time for a Partitioner.

    Exceptions thrown

    java.lang.IllegalArgumentException when an existing interval includes any value less than or equal to v or when v is outside the range min–max.

  10. def addPartitionLessThan(v: T): Unit

    Add a partition for when Record value x < v.

    Add a partition for when Record value x < v. Since partitions may not overlap, this method and addPartitionLessEquals may only be called one time for a Partitioner.

    Exceptions thrown

    java.lang.IllegalArgumentException when an existing interval includes any value less than v, when v is outside the range min–max, or when v equals min.

  11. def addPartitionRange(lower: T, lowerIncluded: Boolean, upper: T, upperIncluded: Boolean): Unit

    Add a partition for when Record value x is within the range lower to upper where lower and/or upper may be excluded from the range by specifying the lowerIncluded and/or upperIncluded parameters as false.

    Add a partition for when Record value x is within the range lower to upper where lower and/or upper may be excluded from the range by specifying the lowerIncluded and/or upperIncluded parameters as false.

    Exceptions thrown

    java.lang.IllegalArgumentException when an existing interval includes any value that would also be included in this interval, when lower or upper are outside the range min–max, when lower is greater than upper, or when either Boolean parameter is false and lower equals upper.

  12. def addPartitionRange(lower: T, upper: T): Unit

    Add a partition for when Record value x satifies lower <= x <= upper.

    Add a partition for when Record value x satifies lower <= x <= upper.

    Exceptions thrown

    java.lang.IllegalArgumentException when an existing interval includes any value between lower and upper inclusive or when lower or upper are outside the range min–max.

  13. def addSinglePartitionEqualsAny(s: Seq[T]): Unit

    Add a single partition for when Record value x equals any value in the list.

    Add a single partition for when Record value x equals any value in the list.

    Exceptions thrown

    java.lang.IllegalArgumentException when an existing interval includes any value in s or when v is outside the range min–max.

    See also

    addMultiplePartitionsEqualsAny for a method that creates multiple partitions: one for each value in a sequence

  14. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  15. def checkSanity(verbose: Boolean): Boolean

    Checks that the ranges in the Partition are sane (i.e., that they do not overlap) and returns true if they do not overlap and false when they do.

    Checks that the ranges in the Partition are sane (i.e., that they do not overlap) and returns true if they do not overlap and false when they do.

    verbose

    If true, print each range as it is examined, any holes found in the Partitions, and ranges that overlap.

  16. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  17. def coverRanges(): Unit

    Adds multiple range rules to this Partitioner for all values that are not currently covered by an existing partition so that every possible value between min and max is covered.

    Adds multiple range rules to this Partitioner for all values that are not currently covered by an existing partition so that every possible value between min and max is covered. In addtion, causes the Partitioner to create an "is_null=KEY" directory for Records that do not contain the field key.

    If called on an empty Partitioner, creates a Partition from min to max inclusive.

    See also

    addNotIn for an alternative way to handle remaining values

  18. val delim: String

    In the directory name generated by this partitoiner, the delimiter between the operator, field name, and argument(s)

    In the directory name generated by this partitoiner, the delimiter between the operator, field name, and argument(s)

    Definition Classes
    Partitioner
  19. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  20. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  21. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  22. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  23. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  24. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  25. val logger: Logger
    Attributes
    protected
    Definition Classes
    StrictLogging
  26. val max_path_component: Int

    The maximum length of a path component that the file system supports.

    The maximum length of a path component that the file system supports.

    Definition Classes
    Partitioner
  27. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  28. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  29. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  30. def pathForFilename(filename: String): Option[String]

    Returns the path component from the file named 'filename' based on this partition as an Option.

    Returns the path component from the file named 'filename' based on this partition as an Option. Returns None there is not a specific component for the field's value. The return value when the field key is not present in the Record depends on the value of useNullComponent: returns None when it is false and Some("is_null${delim}${encodedKey}") when it is true.

    Definition Classes
    NumericPartitioner → Partitioner
  31. def pathForRecord(rec: Record): Option[String]

    Returns the path component for the Record rec based on this partition as an Option.

    Returns the path component for the Record rec based on this partition as an Option. Returns None there is not a specific component for the field's value. The return value when the field key is not present in the Record depends on the value of useNullComponent: returns None when it is false and Some(s"is_null${delim}${encodedKey}") when it is true.

    Definition Classes
    NumericPartitioner → Partitioner
  32. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  33. def toString(): String
    Definition Classes
    AnyRef → Any
  34. var useNullComponent: Boolean

    Whether a missing value in the record is recorded by an "is_null=FIELD" directory component.

  35. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  36. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  37. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Deprecated Value Members

  1. def addPartitionEqualsAny(s: Seq[T]): Unit
    Annotations
    @deprecated
    Deprecated

    (Since version 1.2.2) Replace with addMultiplePartitionsEqualsAny

Inherited from StrictLogging

Inherited from Partitioner

Inherited from AnyRef

Inherited from Any

Ungrouped