sealed abstract class NumericPartitioner[T] extends Partitioner with StrictLogging
NumericPartitioner supports partitioning based on the numeric
field named key in a Record,
where each partition represents records where that field either
has a specific (single) value or falls with a range of values.
NumericPartitioner is the parent class of other classes for specific numeric types (such as 16-bit integers and 32-bit floating point numbers). These subclasses must specify a numeric type and the allowable range of values for that type.
- T
The type of the number
- Alphabetic
- By Inheritance
- NumericPartitioner
- StrictLogging
- Partitioner
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- def addMultiplePartitionsEqualsAny(s: Seq[T]): Unit
Add multiple partitions for when Record value
xequals any value in the list.Add multiple partitions for when Record value
xequals any value in the list. This is a convenience function that invokes addPartitionEquals for each value ins.- Exceptions thrown
java.lang.IllegalArgumentExceptionwhen an existing interval includes any value insor whenvis outside the rangemin–max.- See also
addSinglePartitionEqualsAny for a method that puts multiple values in a single partition
- def addNotIn(): Unit
Adds a single additional rule to this Partitioner that covers all values that are not currently covered by an existing partition.
Adds a single additional rule to this Partitioner that covers all values that are not currently covered by an existing partition. This method may only be used when the Partitioner does not contain any ranges; that is, when only addPartitionEquals, addMultiplePartitionsEqualsAny, and addSinglePartitionEqualsAny have been used.
- Exceptions thrown
java.lang.RuntimeExceptionwhen the resulting path component length would be larger than max_path_component, when the Partitioner contains ranges, or when the Partitioner is empty.- See also
coverRanges for an alternative way to handle remaining values
- def addPartitionEquals(v: T): Unit
Add a partition for when Record value
x==v.Add a partition for when Record value
x==v.- Exceptions thrown
java.lang.IllegalArgumentExceptionwhen an existing interval includesvor whenvis outside the rangemin–max.
- def addPartitionGreaterEquals(v: T): Unit
Add a partition for when Record value
x>=v.Add a partition for when Record value
x>=v. Since partitions may not overlap, this method and addPartitionGreaterThan may only be called one time for a Partitioner.- Exceptions thrown
java.lang.IllegalArgumentExceptionwhen an existing interval includes any value greater than or equal tovor whenvis outside the rangemin–max.
- def addPartitionGreaterThan(v: T): Unit
Add a partition for when Record value
x>v.Add a partition for when Record value
x>v. Since partitions may not overlap, this method and addPartitionGreaterEquals may only be called one time for a Partitioner.- Exceptions thrown
java.lang.IllegalArgumentExceptionwhen an existing interval includes any value greater thanv, whenvis outside the rangemin–max, or whenvequalsmax.
- def addPartitionLessEquals(v: T): Unit
Add a partition for when Record value
x<=v.Add a partition for when Record value
x<=v. Since partitions may not overlap, this method and addPartitionLessThan may only be called one time for a Partitioner.- Exceptions thrown
java.lang.IllegalArgumentExceptionwhen an existing interval includes any value less than or equal tovor whenvis outside the rangemin–max.
- def addPartitionLessThan(v: T): Unit
Add a partition for when Record value
x<v.Add a partition for when Record value
x<v. Since partitions may not overlap, this method and addPartitionLessEquals may only be called one time for a Partitioner.- Exceptions thrown
java.lang.IllegalArgumentExceptionwhen an existing interval includes any value less thanv, whenvis outside the rangemin–max, or whenvequalsmin.
- def addPartitionRange(lower: T, lowerIncluded: Boolean, upper: T, upperIncluded: Boolean): Unit
Add a partition for when Record value
xis within the rangelowertoupperwherelowerand/oruppermay be excluded from the range by specifying thelowerIncludedand/orupperIncludedparameters asfalse.Add a partition for when Record value
xis within the rangelowertoupperwherelowerand/oruppermay be excluded from the range by specifying thelowerIncludedand/orupperIncludedparameters asfalse.- Exceptions thrown
java.lang.IllegalArgumentExceptionwhen an existing interval includes any value that would also be included in this interval, whenlowerorupperare outside the rangemin–max, whenloweris greater thanupper, or when either Boolean parameter isfalseandlowerequalsupper.
- def addPartitionRange(lower: T, upper: T): Unit
Add a partition for when Record value
xsatifieslower<=x<=upper.Add a partition for when Record value
xsatifieslower<=x<=upper.- Exceptions thrown
java.lang.IllegalArgumentExceptionwhen an existing interval includes any value betweenlowerandupperinclusive or whenlowerorupperare outside the rangemin–max.
- def addSinglePartitionEqualsAny(s: Seq[T]): Unit
Add a single partition for when Record value
xequals any value in the list.Add a single partition for when Record value
xequals any value in the list.- Exceptions thrown
java.lang.IllegalArgumentExceptionwhen an existing interval includes any value insor whenvis outside the rangemin–max.- See also
addMultiplePartitionsEqualsAny for a method that creates multiple partitions: one for each value in a sequence
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def checkSanity(verbose: Boolean): Boolean
Checks that the ranges in the Partition are sane (i.e., that they do not overlap) and returns
trueif they do not overlap andfalsewhen they do.Checks that the ranges in the Partition are sane (i.e., that they do not overlap) and returns
trueif they do not overlap andfalsewhen they do.- verbose
If
true, print each range as it is examined, any holes found in the Partitions, and ranges that overlap.
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- def coverRanges(): Unit
Adds multiple range rules to this Partitioner for all values that are not currently covered by an existing partition so that every possible value between
minandmaxis covered.Adds multiple range rules to this Partitioner for all values that are not currently covered by an existing partition so that every possible value between
minandmaxis covered. In addtion, causes the Partitioner to create an "is_null=KEY" directory for Records that do not contain the fieldkey.If called on an empty Partitioner, creates a Partition from
mintomaxinclusive.- See also
addNotIn for an alternative way to handle remaining values
- val delim: String
In the directory name generated by this partitoiner, the delimiter between the operator, field name, and argument(s)
In the directory name generated by this partitoiner, the delimiter between the operator, field name, and argument(s)
- Definition Classes
- Partitioner
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- val logger: Logger
- Attributes
- protected
- Definition Classes
- StrictLogging
- val max_path_component: Int
The maximum length of a path component that the file system supports.
The maximum length of a path component that the file system supports.
- Definition Classes
- Partitioner
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- def pathForFilename(filename: String): Option[String]
Returns the path component from the file named 'filename' based on this partition as an Option.
Returns the path component from the file named 'filename' based on this partition as an Option. Returns None there is not a specific component for the field's value. The return value when the field
keyis not present in the Record depends on the value ofuseNullComponent: returns None when it isfalseandSome("is_null${delim}${encodedKey}")when it istrue.- Definition Classes
- NumericPartitioner → Partitioner
- def pathForRecord(rec: Record): Option[String]
Returns the path component for the Record
recbased on this partition as an Option.Returns the path component for the Record
recbased on this partition as an Option. Returns None there is not a specific component for the field's value. The return value when the fieldkeyis not present in the Record depends on the value ofuseNullComponent: returns None when it isfalseandSome(s"is_null${delim}${encodedKey}")when it istrue.- Definition Classes
- NumericPartitioner → Partitioner
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- var useNullComponent: Boolean
Whether a missing value in the record is recorded by an "is_null=FIELD" directory component.
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()