Packages

trait Partitioning extends AnyRef

Describes how an operator's output is split across partitions. It has 2 major properties:

  1. number of partitions. 2. if it can satisfy a given distribution.
Linear Supertypes
AnyRef, Any
Known Subclasses
BroadcastPartitioning, HashPartitioning, PartitioningCollection, RangePartitioning, RoundRobinPartitioning, SinglePartition, UnknownPartitioning
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. Partitioning
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract val numPartitions: Int

    Returns the number of partitions that the data is split across

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  12. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  13. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  14. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  15. final def satisfies(required: Distribution): Boolean

    Returns true iff the guarantees made by this Partitioning are sufficient to satisfy the partitioning scheme mandated by the required Distribution, i.e.

    Returns true iff the guarantees made by this Partitioning are sufficient to satisfy the partitioning scheme mandated by the required Distribution, i.e. the current dataset does not need to be re-partitioned for the required Distribution (it is possible that tuples within a partition need to be reorganized).

    A Partitioning can never satisfy a Distribution if its numPartitions does't match Distribution.requiredNumPartitions.

  16. def satisfies0(required: Distribution): Boolean

    The actual method that defines whether this Partitioning can satisfy the given Distribution, after the numPartitions check.

    The actual method that defines whether this Partitioning can satisfy the given Distribution, after the numPartitions check.

    By default a Partitioning can satisfy UnspecifiedDistribution, and AllTuples if the Partitioning only have one partition. Implementations can also overwrite this method with special logic.

    Attributes
    protected
  17. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  18. def toString(): String
    Definition Classes
    AnyRef → Any
  19. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  20. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  21. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from AnyRef

Inherited from Any

Ungrouped