ai.databand.spark

Class DbndSparkListener

java.lang.Object

org.apache.spark.scheduler.SparkListener

ai.databand.spark.DbndSparkListener

All Implemented Interfaces:

org.apache.spark.scheduler.SparkListenerInterface

public class DbndSparkListener
extends org.apache.spark.scheduler.SparkListener

Spark listener to report all spark-generated metrics for stages to Databand. This class is safe to use in any circumstances and won't break client code.

Constructor Summary

Constructors

Constructor and Description
DbndSparkListener()
DbndSparkListener(DbndWrapper dbnd)

Method Summary

Modifier and Type	Method and Description
protected void	extractIoInfo(org.apache.spark.sql.execution.SparkPlanInfo plan) input location path read schema data format
void	onOtherEvent(org.apache.spark.scheduler.SparkListenerEvent event)
void	onStageCompleted(org.apache.spark.scheduler.SparkListenerStageCompleted stageCompleted)

Methods inherited from class org.apache.spark.scheduler.SparkListener

onApplicationEnd, onApplicationStart, onBlockManagerAdded, onBlockManagerRemoved, onBlockUpdated, onEnvironmentUpdate, onExecutorAdded, onExecutorBlacklisted, onExecutorBlacklistedForStage, onExecutorMetricsUpdate, onExecutorRemoved, onExecutorUnblacklisted, onJobEnd, onJobStart, onNodeBlacklisted, onNodeBlacklistedForStage, onNodeUnblacklisted, onSpeculativeTaskSubmitted, onStageSubmitted, onTaskEnd, onTaskGettingResult, onTaskStart, onUnpersistRDD

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DbndSparkListener

public DbndSparkListener(DbndWrapper dbnd)

DbndSparkListener

public DbndSparkListener()

Method Detail

onOtherEvent

public void onOtherEvent(org.apache.spark.scheduler.SparkListenerEvent event)

Specified by:

onOtherEvent in interface org.apache.spark.scheduler.SparkListenerInterface

Overrides:

onOtherEvent in class org.apache.spark.scheduler.SparkListener

extractIoInfo

protected void extractIoInfo(org.apache.spark.sql.execution.SparkPlanInfo plan)

input location path read schema data format

Parameters:

plan -

onStageCompleted

public void onStageCompleted(org.apache.spark.scheduler.SparkListenerStageCompleted stageCompleted)

Specified by:

onStageCompleted in interface org.apache.spark.scheduler.SparkListenerInterface

Overrides:

onStageCompleted in class org.apache.spark.scheduler.SparkListener