eus.ixa.ixa.pipe.ml

Class SequenceLabelerTrainer

java.lang.Object

eus.ixa.ixa.pipe.ml.SequenceLabelerTrainer

public class SequenceLabelerTrainer
extends Object

Trainer based on Apache OpenNLP Machine Learning API. This class creates a feature set based on the features activated in the trainParams.properties file:

1. Window: specify left and right window lengths.
2. TokenFeatures: tokens as features in a window length.
3. TokenClassFeatures: token shape features in a window length.
4. WordShapeSuperSenseFeatures: token shape features from Ciaramita and Altun (2006).
5. OutcomePriorFeatures: take into account previous outcomes.
6. PreviousMapFeatures: add features based on tokens and previous decisions.
7. SentenceFeatures: add beginning and end of sentence words.
8. PrefixFeatures: first 4 characters in current token.
9. SuffixFeatures: last 4 characters in current token.
10. BigramClassFeatures: bigrams of tokens and token class.
11. TrigramClassFeatures: trigrams of token and token class.
12. FourgramClassFeatures: fourgrams of token and token class.
13. FivegramClassFeatures: fivegrams of token and token class.
14. CharNgramFeatures: character ngram features of current token.
15. DictionaryFeatures: check if current token appears in some gazetteer.
16. ClarkClusterFeatures: use the clustering class of a token as a feature.
17. BrownClusterFeatures: use brown clusters as features for each feature containing a token.
18. Word2VecClusterFeatures: use the word2vec clustering class of a token as a feature.
19. POSTagModelFeatures: use pos tags, pos tag class as features.
20. LemmaModelFeatures: use lemma as features.
21. LemmaDictionaryFeatures: use lemma from a dictionary as features.
22. MFSFeatures: Most Frequent sense feature.
23. SuperSenseFeatures: Ciaramita and Altun (2006) features for super sense tagging.
24. POSBaselineFeatures: train a baseline POS tagger.
25. LemmaBaselineFeatures: train a baseline Lemmatizer.
26. ChunkBaselineFeatures: train a baseline chunker.

Version:

2016-05-06

Author:

ragerri

Constructor Summary

Constructors

Constructor and Description
SequenceLabelerTrainer(opennlp.tools.util.TrainingParameters params) Construct a trainer with training and test data, and with options for language, beamsize for decoding, sequence codec and corpus format (conll or opennlp).

Method Summary

Modifier and Type	Method and Description
void	createSequenceLabelerFactory(opennlp.tools.util.TrainingParameters params) Create SequenceLabelerFactory with custom features.
String	getSequenceCodec() Get the Sequence codec.
SequenceLabelerFactory	getSequenceLabelerFactory() Get the features which are implemented in each of the trainers extending this class.
static opennlp.tools.util.ObjectStream<SequenceLabelSample>	getSequenceStream(String inputData, String clearFeatures, String aCorpusFormat) Getting the stream with the right corpus format.
void	setSequenceCodec(String aSeqCodec) Set the sequence codec.
SequenceLabelerFactory	setSequenceLabelerFactory(SequenceLabelerFactory tokenNameFinderFactory)
SequenceLabelerModel	train(opennlp.tools.util.TrainingParameters params)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SequenceLabelerTrainer

public SequenceLabelerTrainer(opennlp.tools.util.TrainingParameters params)
                       throws IOException

Construct a trainer with training and test data, and with options for language, beamsize for decoding, sequence codec and corpus format (conll or opennlp).

Parameters:

params - the training parameters

Throws:

IOException - io exception

Method Detail

createSequenceLabelerFactory

public void createSequenceLabelerFactory(opennlp.tools.util.TrainingParameters params)
                                  throws IOException

Create SequenceLabelerFactory with custom features.

Parameters:

params - the parameter training file

Throws:

IOException - if io error

train

public final SequenceLabelerModel train(opennlp.tools.util.TrainingParameters params)

getSequenceStream

public static opennlp.tools.util.ObjectStream<SequenceLabelSample> getSequenceStream(String inputData,
                                                                                     String clearFeatures,
                                                                                     String aCorpusFormat)
                                                                              throws IOException

Getting the stream with the right corpus format.

Parameters:

inputData - the input data

clearFeatures - clear the features

aCorpusFormat - the corpus format

Returns:

the stream from the several corpus formats

Throws:

IOException - the io exception

getSequenceLabelerFactory

public final SequenceLabelerFactory getSequenceLabelerFactory()

Get the features which are implemented in each of the trainers extending this class.

Returns:

the features

setSequenceLabelerFactory

public final SequenceLabelerFactory setSequenceLabelerFactory(SequenceLabelerFactory tokenNameFinderFactory)

getSequenceCodec

public final String getSequenceCodec()

Get the Sequence codec.

Returns:

the sequence codec

setSequenceCodec

public final void setSequenceCodec(String aSeqCodec)

Set the sequence codec.

Parameters:

aSeqCodec - the sequence codec to be set