Class TestingProbe<T>

  • All Implemented Interfaces:
    Serializable

    public class TestingProbe<T>
    extends Object
    implements Serializable
    Allows to monitor a pipeline. Provides a
    • an identity transformation that must be inserted into a pipeline
    • a counter metric that counts the number of elements that are processed by the transformation
    • a distribution metrics that tracks the minimum and maximum time when processing took place in the transformation
    The metrics can be used to calculate the processing rate at the monitored location.

    Code copied from org.apache.beam.sdk.nexmark.Monitor and org.apache.beam.sdk.testutils.metrics.MetricsReader with slight adaptions.

    See Also:
    Serialized Form
    • Field Detail

      • namespace

        public final String namespace
      • elementCounterName

        public final String elementCounterName
      • processingTimeDistributionName

        public final String processingTimeDistributionName
    • Constructor Detail

      • TestingProbe

        public TestingProbe​(String namespace,
                            String prefix)
    • Method Detail

      • getTransform

        public org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<? extends T>,​org.apache.beam.sdk.values.PCollection<T>> getTransform()
      • getElementCount

        public long getElementCount​(org.apache.beam.sdk.PipelineResult result)
        Return the current value for the element counter, or -1 if can't be retrieved. Note this uses only attempted metrics because some runners don't support committed metrics.
      • getStartProcessingTime

        public long getStartProcessingTime​(org.apache.beam.sdk.PipelineResult result)
        Return start time metric by counting the difference between "now" and min value from a distribution metric.
      • getEndProcessingTime

        public long getEndProcessingTime​(org.apache.beam.sdk.PipelineResult result)
      • takeSnapshot

        public TestingProbe.Snapshot takeSnapshot​(org.apache.beam.sdk.PipelineResult result)
      • getElementCount

        public static long getElementCount​(String namespace,
                                           String metricName,
                                           org.apache.beam.sdk.PipelineResult result)
        Return the current value for the element counter, or -1 if can't be retrieved. Note this uses only attempted metrics because some runners don't support committed metrics.