AnnotationStatistics (ClearTK TimeML 2.0.0 API)

java.lang.Object
- org.cleartk.eval.AnnotationStatistics<OUTCOME_TYPE>

All Implemented Interfaces:

Serializable
```
public class AnnotationStatistics<OUTCOME_TYPE extends Comparable<? super OUTCOME_TYPE>>
extends Object
implements Serializable
```
Stores statistics for comparing Annotations extracted by a system to gold Annotations.
Copyright (c) 2012, Regents of the University of Colorado
All rights reserved.

Author:

Steven Bethard

See Also:
Serialized Form

Constructor Summary

Constructors
Constructor and Description
`AnnotationStatistics()` Create an AnnotationStatistics that compares `Annotation`s based on their begin and end offsets, plus a `Feature` of the `Annotation` that represents the outcome or label.

Method Summary

Methods
Modifier and Type	Method and Description
`<ANNOTATION_TYPE extends Annotation> void`	`add(Collection<? extends ANNOTATION_TYPE> referenceAnnotations, Collection<? extends ANNOTATION_TYPE> predictedAnnotations)` Update the statistics, comparing the reference annotations to the predicted annotations.
`<ANNOTATION_TYPE,SPAN_TYPE> void`	`add(Collection<? extends ANNOTATION_TYPE> referenceAnnotations, Collection<? extends ANNOTATION_TYPE> predictedAnnotations, Function<ANNOTATION_TYPE,SPAN_TYPE> annotationToSpan, Function<ANNOTATION_TYPE,OUTCOME_TYPE> annotationToOutcome)` Update the statistics, comparing the reference annotations to the predicted annotations.
`void`	`addAll(AnnotationStatistics<OUTCOME_TYPE> that)` Adds all the statistics collected by another AnnotationStatistics to this one.
`static <OUTCOME_TYPE extends Comparable<? super OUTCOME_TYPE>> AnnotationStatistics<OUTCOME_TYPE>`	`addAll(Iterable<AnnotationStatistics<OUTCOME_TYPE>> statistics)` Add all statistics together.
`static <ANNOTATION_TYPE extends TOP> Function<ANNOTATION_TYPE,String>`	`annotationToFeatureValue(String featureName)` Creates a `Function` that extracts a feature value from a `TOP`.
`static <ANNOTATION_TYPE,OUTCOME_TYPE> Function<ANNOTATION_TYPE,OUTCOME_TYPE>`	`annotationToNull()` Creates a `Function` that always returns null.
`static <ANNOTATION_TYPE extends Annotation> Function<ANNOTATION_TYPE,org.cleartk.eval.AnnotationStatistics.Span>`	`annotationToSpan()` Creates a `Function` that converts an `Annotation` into a hashable representation of its begin and end offsets.
`ConfusionMatrix<OUTCOME_TYPE>`	`confusions()` Returns the `ConfusionMatrix` tabulating reference outcomes matched to predicted outcomes.
`int`	`countCorrectOutcomes()`
`int`	`countCorrectOutcomes(OUTCOME_TYPE outcome)`
`int`	`countFalseNegatives(OUTCOME_TYPE... positiveOutcomes)`
`int`	`countFalsePositives(OUTCOME_TYPE... positiveOutcomes)`
`int`	`countPredictedOutcomes()`
`int`	`countPredictedOutcomes(OUTCOME_TYPE outcome)`
`int`	`countReferenceOutcomes()`
`int`	`countReferenceOutcomes(OUTCOME_TYPE outcome)`
`int`	`countTrueNegatives(OUTCOME_TYPE... positiveOutcomes)`
`int`	`countTruePositives(OUTCOME_TYPE... positiveOutcomes)`
`double`	`f(double beta)`
`double`	`f(double beta, OUTCOME_TYPE outcome)`
`double`	`f1()`
`double`	`f1(OUTCOME_TYPE outcome)`
`double`	`precision()`
`double`	`precision(OUTCOME_TYPE outcome)`
`double`	`recall()`
`double`	`recall(OUTCOME_TYPE outcome)`
`String`	`toString()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Constructor Detail
  - AnnotationStatistics
```
public AnnotationStatistics()
```
    Create an AnnotationStatistics that compares Annotations based on their begin and end offsets, plus a Feature of the Annotation that represents the outcome or label.
- Method Detail
  - add
```
public <ANNOTATION_TYPE extends Annotation> void add(Collection<? extends ANNOTATION_TYPE> referenceAnnotations,
                                            Collection<? extends ANNOTATION_TYPE> predictedAnnotations)
```
    Update the statistics, comparing the reference annotations to the predicted annotations. Annotations are considered to match if they have the same character offsets in the text. All outcomes (e.g. as returned in confusions()) will be null.
    
    Parameters:
    referenceAnnotations - The reference annotations, typically identified by humans.
    predictedAnnotations - The predicted annotations, typically identified by a model.
  - add
```
public <ANNOTATION_TYPE,SPAN_TYPE> void add(Collection<? extends ANNOTATION_TYPE> referenceAnnotations,
                                   Collection<? extends ANNOTATION_TYPE> predictedAnnotations,
                                   Function<ANNOTATION_TYPE,SPAN_TYPE> annotationToSpan,
                                   Function<ANNOTATION_TYPE,OUTCOME_TYPE> annotationToOutcome)
```
    Update the statistics, comparing the reference annotations to the predicted annotations. Annotations are considered to match if they have the same span (according to annotationToSpan) and if they have the same outcome (according to annotationToOutcome).
    
    Parameters:
    referenceAnnotations - The reference annotations, typically identified by humans.
    predictedAnnotations - The predicted annotations, typically identified by a model.
    annotationToSpan - A function that defines how to convert an annotation into a hashable object that represents the span of that annotation. The annotationToSpan() method provides an example function that could be used here.
    annotationToOutcome - A function that defines how to convert an annotation into an object that represents the outcome (or "label") assigned to that annotation. The annotationToFeatureValue(String) method provides a sample function that could be used here.
  - addAll
```
public void addAll(AnnotationStatistics<OUTCOME_TYPE> that)
```
    Adds all the statistics collected by another AnnotationStatistics to this one.
    
    Parameters:
    that - The other statistics that should be added to this one.
  - addAll
```
public static <OUTCOME_TYPE extends Comparable<? super OUTCOME_TYPE>> AnnotationStatistics<OUTCOME_TYPE> addAll(Iterable<AnnotationStatistics<OUTCOME_TYPE>> statistics)
```
    Add all statistics together. This is often useful for combining individual fold statistics that result from methods like Evaluation_ImplBase.crossValidation(List, int).
    
    Parameters:
    statistics - The sequence of statistics that should be combined.
    
    Returns:
    The combination of all the individual statistics.
  - annotationToFeatureValue
```
public static <ANNOTATION_TYPE extends TOP> Function<ANNOTATION_TYPE,String> annotationToFeatureValue(String featureName)
```
    Creates a Function that extracts a feature value from a TOP. The Function created by this method is suitable for passing to the second Function argument of add(Collection, Collection, Function, Function).
    
    Parameters:
    featureName - The name of the feature whose value is to be extracted.
  - annotationToNull
```
public static <ANNOTATION_TYPE,OUTCOME_TYPE> Function<ANNOTATION_TYPE,OUTCOME_TYPE> annotationToNull()
```
    Creates a Function that always returns null. This may be useful when only the span of the offset is important, but you still need to pass in the final argument of add(Collection, Collection, Function, Function).
  - annotationToSpan
```
public static <ANNOTATION_TYPE extends Annotation> Function<ANNOTATION_TYPE,org.cleartk.eval.AnnotationStatistics.Span> annotationToSpan()
```
    Creates a Function that converts an Annotation into a hashable representation of its begin and end offsets. The Function created by this method is suitable for passing to the first Function argument of add(Collection, Collection, Function, Function).
  - confusions
```
public ConfusionMatrix<OUTCOME_TYPE> confusions()
```
    Returns the ConfusionMatrix tabulating reference outcomes matched to predicted outcomes.
    
    Returns:
    The confusion matrix.
  - countCorrectOutcomes
```
public int countCorrectOutcomes()
```
  - countCorrectOutcomes
```
public int countCorrectOutcomes(OUTCOME_TYPE outcome)
```
  - countFalseNegatives
```
public int countFalseNegatives(OUTCOME_TYPE... positiveOutcomes)
```
  - countFalsePositives
```
public int countFalsePositives(OUTCOME_TYPE... positiveOutcomes)
```
  - countPredictedOutcomes
```
public int countPredictedOutcomes()
```
  - countPredictedOutcomes
```
public int countPredictedOutcomes(OUTCOME_TYPE outcome)
```
  - countReferenceOutcomes
```
public int countReferenceOutcomes()
```
  - countReferenceOutcomes
```
public int countReferenceOutcomes(OUTCOME_TYPE outcome)
```
  - countTrueNegatives
```
public int countTrueNegatives(OUTCOME_TYPE... positiveOutcomes)
```
  - countTruePositives
```
public int countTruePositives(OUTCOME_TYPE... positiveOutcomes)
```
  - f
```
public double f(double beta)
```
  - f
```
public double f(double beta,
       OUTCOME_TYPE outcome)
```
  - f1
```
public double f1()
```
  - f1
```
public double f1(OUTCOME_TYPE outcome)
```
  - precision
```
public double precision()
```
  - precision
```
public double precision(OUTCOME_TYPE outcome)
```
  - recall
```
public double recall()
```
  - recall
```
public double recall(OUTCOME_TYPE outcome)
```
  - toString
```
public String toString()
```
    Overrides:
    
    toString in class Object

Class AnnotationStatistics<OUTCOME_TYPE extends Comparable<? super OUTCOME_TYPE>>

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

AnnotationStatistics

Method Detail

add

add

addAll

addAll

annotationToFeatureValue

annotationToNull

annotationToSpan

confusions

countCorrectOutcomes

countCorrectOutcomes

countFalseNegatives

countFalsePositives

countPredictedOutcomes

countPredictedOutcomes

countReferenceOutcomes

countReferenceOutcomes

countTrueNegatives

countTruePositives

f

f

f1

f1

precision

precision

recall

recall

toString