ai.databand.spark

Class ActiveJobTracker

java.lang.Object

ai.databand.spark.ActiveJobTracker

public class ActiveJobTracker
extends java.lang.Object

Constructor Summary

Constructors

Constructor and Description
ActiveJobTracker()

Method Summary

Modifier and Type	Method and Description
static java.util.List<org.apache.spark.rdd.RDD<?>>	allRdds(org.apache.spark.rdd.RDD<?> rdd) Recursively extract all rdds from job.
static java.util.Optional<SparkIOSource>	extractDataFromJdbcRdd(org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD rdd)
static void	track(org.apache.spark.scheduler.ActiveJob job) Extract IO information from Spark ActiveJob.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ActiveJobTracker

public ActiveJobTracker()

Method Detail

track

public static void track(org.apache.spark.scheduler.ActiveJob job)

Extract IO information from Spark ActiveJob.

Parameters:

job - Spark job

extractDataFromJdbcRdd

public static java.util.Optional<SparkIOSource> extractDataFromJdbcRdd(org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD rdd)

allRdds

public static java.util.List<org.apache.spark.rdd.RDD<?>> allRdds(org.apache.spark.rdd.RDD<?> rdd)

Recursively extract all rdds from job.

Parameters:

rdd - top-level rdd

Returns:

list of all downstream rdds