final class SecondarySortGroupAndSortByPairRDDFunctions[K, V] extends Serializable
- Alphabetic
- By Inheritance
- SecondarySortGroupAndSortByPairRDDFunctions
- Serializable
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
- new SecondarySortGroupAndSortByPairRDDFunctions(rdd: RDD[(K, V)])(implicit arg0: Ordering[K], arg1: ClassTag[K], arg2: ClassTag[V])
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- def foldLeftByKeySortedBy[S, A](startValue: A, op: (A, V) => A, sortBy: (V) => S)(implicit arg0: Ordering[S]): RDD[(K, A)]
Groups by key and applies a binary operation using foldLeft over the values sorted by some implicit ordering
Groups by key and applies a binary operation using foldLeft over the values sorted by some implicit ordering
- A
the result type of the folding operation
- startValue
the start value for the fold
- op
the binary operation for folding
- sortBy
how to sort values
- returns
PairRDD with keys and values, where values are the result of applying foldLeft across the sorted values
- def foldLeftByKeySortedBy[S, A](startValue: A, op: (A, V) => A, sortBy: (V) => S, numPartitions: Int)(implicit arg0: Ordering[S]): RDD[(K, A)]
Groups by key and applies a binary operation using foldLeft over the values sorted by some implicit ordering
Groups by key and applies a binary operation using foldLeft over the values sorted by some implicit ordering
- A
the result type of the folding operation
- startValue
the start value for the fold
- op
the binary operation for folding
- sortBy
how to sort values
- numPartitions
the number of partitions for shuffling
- returns
PairRDD with keys and values, where values are the result of applying foldLeft across the sorted values
- def foldLeftByKeySortedBy[S, A](startValue: A, op: (A, V) => A, sortBy: (V) => S, partitioner: Partitioner)(implicit arg0: Ordering[S]): RDD[(K, A)]
Groups by key and applies a binary operation using foldLeft over the values sorted by some implicit ordering
Groups by key and applies a binary operation using foldLeft over the values sorted by some implicit ordering
- A
the result type of the folding operation
- startValue
the start value for the fold
- op
the binary operation for folding
- sortBy
how to sort values
- partitioner
the partitioner for shuffling
- returns
PairRDD with keys and values, where values are the result of applying foldLeft across the sorted values
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def groupByKeyAndSortBy[A](sortBy: (V) => A)(implicit arg0: Ordering[A]): RDD[(K, Iterable[V])]
Groups by key and sorts the values by some implicit ordering
Groups by key and sorts the values by some implicit ordering
- sortBy
how to sort values
- returns
a PairRDD of keys and sorted values
- def groupByKeyAndSortBy[A](sortBy: (V) => A, numPartitions: Int)(implicit arg0: Ordering[A]): RDD[(K, Iterable[V])]
Groups by key and sorts the values by some implicit ordering
Groups by key and sorts the values by some implicit ordering
- sortBy
how to sort values
- numPartitions
the number of partitions for shuffling
- returns
PairRDD of keys and sorted values
- def groupByKeyAndSortBy[A](sortBy: (V) => A, partitioner: Partitioner)(implicit arg0: Ordering[A]): RDD[(K, Iterable[V])]
Groups by key and sorts the values by some implicit ordering
Groups by key and sorts the values by some implicit ordering
- sortBy
how to sort values
- partitioner
the partitioner for shuffling
- returns
PairRDD of keys and sorted values
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def mapValuesWithKeyedPreparedResourceSortedBy[R, R1, S, A](resources: RDD[(K, R)], prepareResource: (R) => R1, op: (R1, V) => A, sortBy: (V) => S)(implicit arg0: ClassTag[R], arg1: Ordering[S]): RDD[(K, A)]
Applies op to every value with some resource, where values and resources share the same key.
Applies op to every value with some resource, where values and resources share the same key. This allows you to send data to executors based on key, so that: (1) the entire set of resources are not held in memory on all executors; and, (2) a specific resource is not duplicated; it is reused for all corresponding data values One example usage might be when conducting a geospatial operation. If the keys indicate a geographic area, and the value contains geospatial resources in that geographic area, one can apply a method using geospatially local resources to all values while reducing data duplication and shuffling.
- R
the type of resources being used
- A
the type returned by applying the operation with the resource to each value
- resources
a PairRDD of keys and resources, where keys are used to determine what data the resource is associated with for the operation. There must be exactly one resource for each key in the RDD this method is applied to
- prepareResource
a function to transform the resource into what will be used in op
- op
the operation to apply to each value. The operation takes a resource and value and returns the transformed value
- sortBy
how to sort values
- returns
PairRDD of values transformed by applying the operation with the appropriate resource
- def mapValuesWithKeyedPreparedResourceSortedBy[R, R1, S, A](resources: RDD[(K, R)], prepareResource: (R) => R1, op: (R1, V) => A, sortBy: (V) => S, numPartitions: Int)(implicit arg0: ClassTag[R], arg1: Ordering[S]): RDD[(K, A)]
Applies op to every value with some resource, where values and resources share the same key.
Applies op to every value with some resource, where values and resources share the same key. This allows you to send data to executors based on key, so that: (1) the entire set of resources are not held in memory on all executors; and, (2) a specific resource is not duplicated; it is reused for all corresponding data values One example usage might be when conducting a geospatial operation. If the keys indicate a geographic area, and the value contains geospatial resources in that geographic area, one can apply a method using geospatially local resources to all values while reducing data duplication and shuffling.
- R
the type of resources being used
- A
the type returned by applying the operation with the resource to each value
- resources
a PairRDD of keys and resources, where keys are used to determine what data the resource is associated with for the operation. There must be exactly one resource for each key in the RDD this method is applied to
- prepareResource
a function to transform the resource into what will be used in op
- op
the operation to apply to each value. The operation takes a resource and value and returns the transformed value
- sortBy
how to sort values
- numPartitions
the number of partitions for shuffling
- returns
PairRDD of values transformed by applying the operation with the appropriate resource
- def mapValuesWithKeyedPreparedResourceSortedBy[R, R1, S, A](resources: RDD[(K, R)], prepareResource: (R) => R1, op: (R1, V) => A, sortBy: (V) => S, partitioner: Partitioner)(implicit arg0: ClassTag[R], arg1: Ordering[S]): RDD[(K, A)]
Applies op to every value with some resource, where values and resources share the same key.
Applies op to every value with some resource, where values and resources share the same key. This allows you to send data to executors based on key, so that: (1) the entire set of resources are not held in memory on all executors; and, (2) a specific resource is not duplicated; it is reused for all corresponding data values One example usage might be when conducting a geospatial operation. If the keys indicate a geographic area, and the value contains geospatial resources in that geographic area, one can apply a method using geospatially local resources to all values while reducing data duplication and shuffling.
- R
the type of resources being used
- A
the type returned by applying the operation with the resource to each value
- resources
a PairRDD of keys and resources, where keys are used to determine what data the resource is associated with for the operation. There must be exactly one resource for each key in the RDD this method is applied to
- prepareResource
a function to transform the resource into what will be used in op
- op
the operation to apply to each value. The operation takes a resource and value and returns the transformed value
- sortBy
how to sort values
- partitioner
the partitioner for shuffling
- returns
PairRDD of values transformed by applying the operation with the appropriate resource
- def mapValuesWithKeyedPreparedResourceSortedBy[R, S, A](resources: RDD[(K, R)], op: (R) => (V) => A, sortBy: (V) => S)(implicit arg0: ClassTag[R], arg1: Ordering[S]): RDD[(K, A)]
Applies op to every value with some resource, where values and resources share the same key.
Applies op to every value with some resource, where values and resources share the same key. This allows you to send data to executors based on key, so that: (1) the entire set of resources are not held in memory on all executors; and, (2) a specific resource is not duplicated; it is reused for all corresponding data values One example usage might be when conducting a geospatial operation. If the keys indicate a geographic area, and the value contains geospatial resources in that geographic area, one can apply a method using geospatially local resources to all values while reducing data duplication and shuffling.
- R
the type of resources being used
- A
the type returned by applying the operation with the resource to each value
- resources
a PairRDD of keys and resources, where keys are used to determine what data the resource is associated with for the operation. There must be exactly one resource for each key in the RDD this method is applied to
- op
The operation to apply to each value. The operation takes a resource and returns a function, which will then be applied to each value.
- sortBy
how to sort values
- returns
PairRDD of values transformed by applying the operation with the appropriate resource
- def mapValuesWithKeyedPreparedResourceSortedBy[R, S, A](resources: RDD[(K, R)], op: (R) => (V) => A, sortBy: (V) => S, numPartitions: Int)(implicit arg0: ClassTag[R], arg1: Ordering[S]): RDD[(K, A)]
Applies op to every value with some resource, where values and resources share the same key.
Applies op to every value with some resource, where values and resources share the same key. This allows you to send data to executors based on key, so that: (1) the entire set of resources are not held in memory on all executors; and, (2) a specific resource is not duplicated; it is reused for all corresponding data values One example usage might be when conducting a geospatial operation. If the keys indicate a geographic area, and the value contains geospatial resources in that geographic area, one can apply a method using geospatially local resources to all values while reducing data duplication and shuffling.
- R
the type of resources being used
- A
the type returned by applying the operation with the resource to each value
- resources
a PairRDD of keys and resources, where keys are used to determine what data the resource is associated with for the operation. There must be exactly one resource for each key in the RDD this method is applied to
- op
The operation to apply to each value. The operation takes a resource and returns a function, which will then be applied to each value.
- sortBy
how to sort values
- numPartitions
the number of partitions for shuffling
- returns
PairRDD of values transformed by applying the operation with the appropriate resource
- def mapValuesWithKeyedPreparedResourceSortedBy[R, S, A](resources: RDD[(K, R)], op: (R) => (V) => A, sortBy: (V) => S, partitioner: Partitioner)(implicit arg0: ClassTag[R], arg1: Ordering[S]): RDD[(K, A)]
Applies op to every value with some resource, where values and resources share the same key.
Applies op to every value with some resource, where values and resources share the same key. This allows you to send data to executors based on key, so that: (1) the entire set of resources are not held in memory on all executors; and, (2) a specific resource is not duplicated; it is reused for all corresponding data values One example usage might be when conducting a geospatial operation. If the keys indicate a geographic area, and the value contains geospatial resources in that geographic area, one can apply a method using geospatially local resources to all values while reducing data duplication and shuffling.
- R
the type of resources being used
- A
the type returned by applying the operation with the resource to each value
- resources
a PairRDD of keys and resources, where keys are used to determine what data the resource is associated with for the operation. There must be exactly one resource for each key in the RDD this method is applied to
- op
The operation to apply to each value. The operation takes a resource and returns a function, which will then be applied to each value.
- sortBy
how to sort values
- partitioner
the partitioner for shuffling
- returns
PairRDD of values transformed by applying the operation with the appropriate resource
- def mapValuesWithKeyedResourceSortedBy[R, S, A](resources: RDD[(K, R)], op: (R, V) => A, sortBy: (V) => S)(implicit arg0: ClassTag[R], arg1: Ordering[S]): RDD[(K, A)]
Applies op to every value with some resource, where values and resources share the same key.
Applies op to every value with some resource, where values and resources share the same key. This allows you to send data to executors based on key, so that: (1) the entire set of resources are not held in memory on all executors; and, (2) a specific resource is not duplicated; it is reused for all corresponding data values One example usage might be when conducting a geospatial operation. If the keys indicate a geographic area, and the value contains geospatial resources in that geographic area, one can apply a method using geospatially local resources to all values while reducing data duplication and shuffling.
- R
the type of resources being used
- A
the type returned by applying the operation with the resource to each value
- resources
a PairRDD of keys and resources, where keys are used to determine what data the resource is associated with for the operation. There must be exactly one resource for each key in the RDD this method is applied to
- op
the operation to apply to each value. Takes a resource and value and returns the transformed value
- sortBy
how to sort values
- returns
PairRDD of values transformed by applying the operation with the appropriate resource
- def mapValuesWithKeyedResourceSortedBy[R, S, A](resources: RDD[(K, R)], op: (R, V) => A, sortBy: (V) => S, numPartitions: Int)(implicit arg0: ClassTag[R], arg1: Ordering[S]): RDD[(K, A)]
Applies op to every value with some resource, where values and resources share the same key.
Applies op to every value with some resource, where values and resources share the same key. This allows you to send data to executors based on key, so that: (1) the entire set of resources are not held in memory on all executors; and, (2) a specific resource is not duplicated; it is reused for all corresponding data values One example usage might be when conducting a geospatial operation. If the keys indicate a geographic area, and the value contains geospatial resources in that geographic area, one can apply a method using geospatially local resources to all values while reducing data duplication and shuffling.
- R
the type of resources being used
- A
the type returned by applying the operation with the resource to each value
- resources
a PairRDD of keys and resources, where keys are used to determine what data the resource is associated with for the operation. There must be exactly one resource for each key in the RDD this method is applied to
- op
the operation to apply to each value. Takes a resource and value and returns the transformed value
- sortBy
how to sort values
- numPartitions
the number of partitions for shuffling
- returns
PairRDD of values transformed by applying the operation with the appropriate resource
- def mapValuesWithKeyedResourceSortedBy[R, S, A](resources: RDD[(K, R)], op: (R, V) => A, sortBy: (V) => S, partitioner: Partitioner)(implicit arg0: ClassTag[R], arg1: Ordering[S]): RDD[(K, A)]
Applies op to every value with some resource, where values and resources share the same key.
Applies op to every value with some resource, where values and resources share the same key. This allows you to send data to executors based on key, so that: (1) the entire set of resources are not held in memory on all executors; and, (2) a specific resource is not duplicated; it is reused for all corresponding data values One example usage might be when conducting a geospatial operation. If the keys indicate a geographic area, and the value contains geospatial resources in that geographic area, one can apply a method using geospatially local resources to all values while reducing data duplication and shuffling.
- R
the type of resources being used
- A
the type returned by applying the operation with the resource to each value
- resources
a PairRDD of keys and resources, where keys are used to determine what data the resource is associated with for the operation. There must be exactly one resource for each key in the RDD this method is applied to
- op
the operation to apply to each value. Takes a resource and value and returns the transformed value
- sortBy
how to sort values
- partitioner
the partitioner for shuffling
- returns
PairRDD of values transformed by applying the operation with the appropriate resource
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()