final class SecondarySortGroupingPairRDDFunctions[K, V] extends Serializable
- Alphabetic
- By Inheritance
- SecondarySortGroupingPairRDDFunctions
- Serializable
- Serializable
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Instance Constructors
- new SecondarySortGroupingPairRDDFunctions(rdd: RDD[(K, V)])(implicit arg0: Ordering[K], arg1: ClassTag[K], arg2: Ordering[V], arg3: ClassTag[V])
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
def
mapValuesWithKeyedPreparedResource[R, R1, A](resources: RDD[(K, R)], prepareResource: (R) ⇒ R1, op: (R1, V) ⇒ A)(implicit arg0: ClassTag[R]): RDD[(K, A)]
Applies op to every value with some resource, where values and resources share the same key.
Applies op to every value with some resource, where values and resources share the same key. This allows you to send data to executors based on key, so that: (1) the entire set of resources are not held in memory on all executors; and, (2) a specific resource is not duplicated; it is reused for all corresponding data values One example usage might be when conducting a geospatial operation. If the keys indicate a geographic area, and the value contains geospatial resources in that geographic area, one can apply a method using geospatially local resources to all values while reducing data duplication and shuffling.
- R
the type of resources being used
- A
the type returned by applying the operation with the resource to each value
- resources
a PairRDD of keys and resources, where keys are used to determine what data the resource is associated with for the operation. There must be exactly one resource for each key in the RDD this method is applied to
- prepareResource
a function to transform the resource into what will be used in op
- op
the operation to apply to each value. The operation takes a resource and value and returns the transformed value
- returns
PairRDD of values transformed by applying the operation with the appropriate resource
-
def
mapValuesWithKeyedPreparedResource[R, R1, A](resources: RDD[(K, R)], prepareResource: (R) ⇒ R1, op: (R1, V) ⇒ A, numPartitions: Int)(implicit arg0: ClassTag[R]): RDD[(K, A)]
Applies op to every value with some resource, where values and resources share the same key.
Applies op to every value with some resource, where values and resources share the same key. This allows you to send data to executors based on key, so that: (1) the entire set of resources are not held in memory on all executors; and, (2) a specific resource is not duplicated; it is reused for all corresponding data values One example usage might be when conducting a geospatial operation. If the keys indicate a geographic area, and the value contains geospatial resources in that geographic area, one can apply a method using geospatially local resources to all values while reducing data duplication and shuffling.
- R
the type of resources being used
- A
the type returned by applying the operation with the resource to each value
- resources
a PairRDD of keys and resources, where keys are used to determine what data the resource is associated with for the operation. There must be exactly one resource for each key in the RDD this method is applied to
- prepareResource
a function to transform the resource into what will be used in op
- op
the operation to apply to each value. The operation takes a resource and value and returns the transformed value
- numPartitions
the number of partitions for shuffling
- returns
PairRDD of values transformed by applying the operation with the appropriate resource
-
def
mapValuesWithKeyedPreparedResource[R, R1, A](resources: RDD[(K, R)], prepareResource: (R) ⇒ R1, op: (R1, V) ⇒ A, partitioner: Partitioner)(implicit arg0: ClassTag[R]): RDD[(K, A)]
Applies op to every value with some resource, where values and resources share the same key.
Applies op to every value with some resource, where values and resources share the same key. This allows you to send data to executors based on key, so that: (1) the entire set of resources are not held in memory on all executors; and, (2) a specific resource is not duplicated; it is reused for all corresponding data values One example usage might be when conducting a geospatial operation. If the keys indicate a geographic area, and the value contains geospatial resources in that geographic area, one can apply a method using geospatially local resources to all values while reducing data duplication and shuffling.
- R
the type of resources being used
- A
the type returned by applying the operation with the resource to each value
- resources
a PairRDD of keys and resources, where keys are used to determine what data the resource is associated with for the operation. There must be exactly one resource for each key in the RDD this method is applied to
- prepareResource
a function to transform the resource into what will be used in op
- op
the operation to apply to each value. The operation takes a resource and value and returns the transformed value
- partitioner
the partitioner for shuffling
- returns
PairRDD of values transformed by applying the operation with the appropriate resource
-
def
mapValuesWithKeyedPreparedResource[R, A](resources: RDD[(K, R)], op: (R) ⇒ (V) ⇒ A)(implicit arg0: ClassTag[R]): RDD[(K, A)]
Applies op to every value with some resource, where values and resources share the same key.
Applies op to every value with some resource, where values and resources share the same key. This allows you to send data to executors based on key, so that: (1) the entire set of resources are not held in memory on all executors; and, (2) a specific resource is not duplicated; it is reused for all corresponding data values One example usage might be when conducting a geospatial operation. If the keys indicate a geographic area, and the value contains geospatial resources in that geographic area, one can apply a method using geospatially local resources to all values while reducing data duplication and shuffling.
- R
the type of resources being used
- A
the type returned by applying the operation with the resource to each value
- resources
a PairRDD of keys and resources, where keys are used to determine what data the resource is associated with for the operation. There must be exactly one resource for each key in the RDD this method is applied to
- op
The operation to apply to each value. The operation takes a resource and returns a function, which will then be applied to each value.
- returns
PairRDD of values transformed by applying the operation with the appropriate resource
-
def
mapValuesWithKeyedPreparedResource[R, A](resources: RDD[(K, R)], op: (R) ⇒ (V) ⇒ A, numPartitions: Int)(implicit arg0: ClassTag[R]): RDD[(K, A)]
Applies op to every value with some resource, where values and resources share the same key.
Applies op to every value with some resource, where values and resources share the same key. This allows you to send data to executors based on key, so that: (1) the entire set of resources are not held in memory on all executors; and, (2) a specific resource is not duplicated; it is reused for all corresponding data values One example usage might be when conducting a geospatial operation. If the keys indicate a geographic area, and the value contains geospatial resources in that geographic area, one can apply a method using geospatially local resources to all values while reducing data duplication and shuffling.
- R
the type of resources being used
- A
the type returned by applying the operation with the resource to each value
- resources
a PairRDD of keys and resources, where keys are used to determine what data the resource is associated with for the operation. There must be exactly one resource for each key in the RDD this method is applied to
- op
The operation to apply to each value. The operation takes a resource and returns a function, which will then be applied to each value.
- numPartitions
the number of partitions for shuffling
- returns
PairRDD of values transformed by applying the operation with the appropriate resource
-
def
mapValuesWithKeyedPreparedResource[R, A](resources: RDD[(K, R)], op: (R) ⇒ (V) ⇒ A, partitioner: Partitioner)(implicit arg0: ClassTag[R]): RDD[(K, A)]
Applies op to every value with some resource, where values and resources share the same key.
Applies op to every value with some resource, where values and resources share the same key. This allows you to send data to executors based on key, so that: (1) the entire set of resources are not held in memory on all executors; and, (2) a specific resource is not duplicated; it is reused for all corresponding data values One example usage might be when conducting a geospatial operation. If the keys indicate a geographic area, and the value contains geospatial resources in that geographic area, one can apply a method using geospatially local resources to all values while reducing data duplication and shuffling.
- R
the type of resources being used
- A
the type returned by applying the operation with the resource to each value
- resources
a PairRDD of keys and resources, where keys are used to determine what data the resource is associated with for the operation. There must be exactly one resource for each key in the RDD this method is applied to
- op
The operation to apply to each value. The operation takes a resource and returns a function, which will then be applied to each value.
- partitioner
the partitioner for shuffling
- returns
PairRDD of values transformed by applying the operation with the appropriate resource
-
def
mapValuesWithKeyedResource[R, A](resources: RDD[(K, R)], op: (R, V) ⇒ A)(implicit arg0: ClassTag[R]): RDD[(K, A)]
Applies op to every value with some resource, where values and resources share the same key.
Applies op to every value with some resource, where values and resources share the same key. This allows you to send data to executors based on key, so that: (1) the entire set of resources are not held in memory on all executors; and, (2) a specific resource is not duplicated; it is reused for all corresponding data values One example usage might be when conducting a geospatial operation. If the keys indicate a geographic area, and the value contains geospatial resources in that geographic area, one can apply a method using geospatially local resources to all values while reducing data duplication and shuffling.
- R
the type of resources being used
- A
the type returned by applying the operation with the resource to each value
- resources
a PairRDD of keys and resources, where keys are used to determine what data the resource is associated with for the operation. There must be exactly one resource for each key in the RDD this method is applied to
- op
the operation to apply to each value. Takes a resource and value and returns the transformed value
- returns
PairRDD of values transformed by applying the operation with the appropriate resource
-
def
mapValuesWithKeyedResource[R, A](resources: RDD[(K, R)], op: (R, V) ⇒ A, numPartitions: Int)(implicit arg0: ClassTag[R]): RDD[(K, A)]
Applies op to every value with some resource, where values and resources share the same key.
Applies op to every value with some resource, where values and resources share the same key. This allows you to send data to executors based on key, so that: (1) the entire set of resources are not held in memory on all executors; and, (2) a specific resource is not duplicated; it is reused for all corresponding data values One example usage might be when conducting a geospatial operation. If the keys indicate a geographic area, and the value contains geospatial resources in that geographic area, one can apply a method using geospatially local resources to all values while reducing data duplication and shuffling.
- R
the type of resources being used
- A
the type returned by applying the operation with the resource to each value
- resources
a PairRDD of keys and resources, where keys are used to determine what data the resource is associated with for the operation. There must be exactly one resource for each key in the RDD this method is applied to
- op
the operation to apply to each value. Takes a resource and value and returns the transformed value
- numPartitions
the number of partitions for shuffling
- returns
PairRDD of values transformed by applying the operation with the appropriate resource
-
def
mapValuesWithKeyedResource[R, A](resources: RDD[(K, R)], op: (R, V) ⇒ A, partitioner: Partitioner)(implicit arg0: ClassTag[R]): RDD[(K, A)]
Applies op to every value with some resource, where values and resources share the same key.
Applies op to every value with some resource, where values and resources share the same key. This allows you to send data to executors based on key, so that: (1) the entire set of resources are not held in memory on all executors; and, (2) a specific resource is not duplicated; it is reused for all corresponding data values One example usage might be when conducting a geospatial operation. If the keys indicate a geographic area, and the value contains geospatial resources in that geographic area, one can apply a method using geospatially local resources to all values while reducing data duplication and shuffling.
- R
the type of resources being used
- A
the type returned by applying the operation with the resource to each value
- resources
a PairRDD of keys and resources, where keys are used to determine what data the resource is associated with for the operation. There must be exactly one resource for each key in the RDD this method is applied to
- op
the operation to apply to each value. Takes a resource and value and returns the transformed value
- partitioner
the partitioner for shuffling
- returns
PairRDD of values transformed by applying the operation with the appropriate resource
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
def
sortedFoldLeftByKey[A](startValue: A, op: (A, V) ⇒ A): RDD[(K, A)]
Groups by key and applies a binary operation using foldLeft over the values sorted by some implicit ordering
Groups by key and applies a binary operation using foldLeft over the values sorted by some implicit ordering
- A
the result type of the folding operation
- startValue
the start value for the fold
- op
the binary operation for folding
- returns
PairRDD with keys and values, where values are the result of applying foldLeft across the sorted values
-
def
sortedFoldLeftByKey[A](startValue: A, op: (A, V) ⇒ A, numPartitions: Int): RDD[(K, A)]
Groups by key and applies a binary operation using foldLeft over the values sorted by some implicit ordering
Groups by key and applies a binary operation using foldLeft over the values sorted by some implicit ordering
- A
the result type of the folding operation
- startValue
the start value for the fold
- op
the binary operation for folding
- numPartitions
the number of partitions for shuffling
- returns
PairRDD with keys and values, where values are the result of applying foldLeft across the sorted values
-
def
sortedFoldLeftByKey[A](startValue: A, op: (A, V) ⇒ A, partitioner: Partitioner): RDD[(K, A)]
Groups by key and applies a binary operation using foldLeft over the values sorted by some implicit ordering
Groups by key and applies a binary operation using foldLeft over the values sorted by some implicit ordering
- A
the result type of the folding operation
- startValue
the start value for the fold
- op
the binary operation for folding
- partitioner
the partitioner for shuffling
- returns
PairRDD with keys and values, where values are the result of applying foldLeft across the sorted values
-
def
sortedGroupByKey: RDD[(K, Iterable[V])]
Groups by key and sorts the values by some implicit ordering
Groups by key and sorts the values by some implicit ordering
- returns
a PairRDD of keys and sorted values
-
def
sortedGroupByKey(numPartitions: Int): RDD[(K, Iterable[V])]
Groups by key and sorts the values by some implicit ordering
Groups by key and sorts the values by some implicit ordering
- numPartitions
the number of partitions for shuffling
- returns
PairRDD of keys and sorted values
-
def
sortedGroupByKey(partitioner: Partitioner): RDD[(K, Iterable[V])]
Groups by key and sorts the values by some implicit ordering
Groups by key and sorts the values by some implicit ordering
- partitioner
the partitioner for shuffling
- returns
PairRDD of keys and sorted values
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()