eus.ixa.ixa.pipe.ml

Class StatisticalSequenceLabeler

java.lang.Object

eus.ixa.ixa.pipe.ml.StatisticalSequenceLabeler

public class StatisticalSequenceLabeler
extends Object

Statistical Sequence Labeling based on Apache OpenNLP Machine Learning API.

Version:

2016-04-01

Author:

ragerri

Constructor Summary

Constructors

Constructor and Description
StatisticalSequenceLabeler(Properties props) Construct a probabilistic sequence labeler.
StatisticalSequenceLabeler(Properties props, SequenceLabelFactory aSeqFactory) Construct a StatisticalSequenceLabeler specifying the factory to be used.
StatisticalSequenceLabeler(String model, String lang) Construct a StatisticalSequenceLabeler specifying the model to be used.
StatisticalSequenceLabeler(String model, String lang, SequenceLabelFactory aSeqFactory) Construct a StatisticalSequenceLabeler specifying the model and the factory to be used.

Method Summary

Modifier and Type	Method and Description
void	clearAdaptiveData() Forgets all adaptive data which was collected during previous calls to one of the find methods.
Span[][]	getAllTags(String[] tokens) Produces a multidimensional array containing all the taggings possible for a given sentence.
List<SequenceLabel>	getLemmaSequences(String[] tokens)
List<SequenceLabel>	getLemmaSequencesFromSpans(String[] tokens, Span[] seqSpans)
com.google.common.collect.ListMultimap<String,String>	getMultipleLemmas(String[] tokens, Span[][] posTags) Takes a sentence with multiple tags alternatives for each word and produces a lemma for each of the word-tag combinations.
List<SequenceLabel>	getSequences(String[] tokens) Produce a list of the SequenceLabel objects classified by the probabilistic model.
List<SequenceLabel>	getSequencesFromSpans(String[] tokens, Span[] seqSpans) Creates a list of SequenceLabel objects from spans and tokens.
Span[]	lemmatizeToSpans(String[] tokens)
Span[]	seqToSpans(String[] tokens) Get array of Spans from a list of tokens.
String[]	seqToStrings(String[] tokens)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StatisticalSequenceLabeler

public StatisticalSequenceLabeler(Properties props)

Construct a probabilistic sequence labeler.

Parameters:

props - the properties to be loaded

StatisticalSequenceLabeler

public StatisticalSequenceLabeler(String model,
                                  String lang)

Construct a StatisticalSequenceLabeler specifying the model to be used.

Parameters:

model - the specific model to be used.

lang - the language

StatisticalSequenceLabeler

public StatisticalSequenceLabeler(Properties props,
                                  SequenceLabelFactory aSeqFactory)

Construct a StatisticalSequenceLabeler specifying the factory to be used.

Parameters:

props - the properties

aSeqFactory - the name factory to construct Name objects

StatisticalSequenceLabeler

public StatisticalSequenceLabeler(String model,
                                  String lang,
                                  SequenceLabelFactory aSeqFactory)

Construct a StatisticalSequenceLabeler specifying the model and the factory to be used.

Parameters:

model - the specific model to be used.

lang - the language

aSeqFactory - the factory

Method Detail

seqToSpans

public final Span[] seqToSpans(String[] tokens)

Get array of Spans from a list of tokens.

Parameters:

tokens - the sentence tokens

Returns:

the array of Sequence Spans

lemmatizeToSpans

public final Span[] lemmatizeToSpans(String[] tokens)

seqToStrings

public final String[] seqToStrings(String[] tokens)

getSequences

public final List<SequenceLabel> getSequences(String[] tokens)

Produce a list of the SequenceLabel objects classified by the probabilistic model. Takes an array of tokens, calls seqToSpans function for probabilistic Sequence Labeling and returns a List of SequenceLabel objects containing the string, the type and the Span.

Parameters:

tokens - an array of tokenized text

Returns:

a List of sequences

getLemmaSequences

public final List<SequenceLabel> getLemmaSequences(String[] tokens)

getSequencesFromSpans

public final List<SequenceLabel> getSequencesFromSpans(String[] tokens,
                                                       Span[] seqSpans)

Creates a list of SequenceLabel objects from spans and tokens.

Parameters:

seqSpans - the sequence spans of a sentence

tokens - the tokens in the sentence

Returns:

a list of SequenceLabel objects

getLemmaSequencesFromSpans

public final List<SequenceLabel> getLemmaSequencesFromSpans(String[] tokens,
                                                            Span[] seqSpans)

getAllTags

public final Span[][] getAllTags(String[] tokens)

Produces a multidimensional array containing all the taggings possible for a given sentence.

Parameters:

tokens - the tokens

Returns:

the array containing for each row the tags

getMultipleLemmas

public com.google.common.collect.ListMultimap<String,String> getMultipleLemmas(String[] tokens,
                                                                               Span[][] posTags)

Takes a sentence with multiple tags alternatives for each word and produces a lemma for each of the word-tag combinations.

Parameters:

tokens - the sentence tokens

posTags - the alternative postags

Returns:

the ordered map containing all the possible tag#lemma values for token

clearAdaptiveData

public final void clearAdaptiveData()

Forgets all adaptive data which was collected during previous calls to one of the find methods. This method is typically called at the end of a document. From Apache OpenNLP documentation: "After every document clearAdaptiveData must be called to clear the adaptive data in the feature generators. Not calling clearAdaptiveData can lead to a sharp drop in the detection rate after a few documents."