eus.ixa.ixa.pipe.nerc.train

Class FixedTrainer

java.lang.Object

eus.ixa.ixa.pipe.nerc.train.AbstractTrainer

eus.ixa.ixa.pipe.nerc.train.FixedTrainer

All Implemented Interfaces:

Trainer

public class FixedTrainer
extends AbstractTrainer

Training sequence labeler based on Apache OpenNLP Machine Learning API. This class creates a feature set based on the features activated in the trainParams.properties file:

1. Window: specify left and right window lengths.
2. TokenFeatures: tokens as features in a window length.
3. TokenClassFeatures: token shape features in a window length.
4. WordShapeSuperSenseFeatures: token shape features from Ciaramita and Altun (2006).
5. OutcomePriorFeatures: take into account previous outcomes.
6. PreviousMapFeatures: add features based on tokens and previous decisions.
7. SentenceFeatures: add beginning and end of sentence words.
8. PrefixFeatures: first 4 characters in current token.
9. SuffixFeatures: last 4 characters in current token.
10. BigramClassFeatures: bigrams of tokens and token class.
11. TrigramClassFeatures: trigrams of token and token class.
12. FourgramClassFeatures: fourgrams of token and token class.
13. FivegramClassFeatures: fivegrams of token and token class.
14. CharNgramFeatures: character ngram features of current token.
15. DictionaryFeatures: check if current token appears in some gazetteer.
16. ClarkClusterFeatures: use the clustering class of a token as a feature.
17. BrownClusterFeatures: use brown clusters as features for each feature containing a token.
18. Word2VecClusterFeatures: use the word2vec clustering class of a token as a feature.
19. MorphoFeatures: use pos tags, pos tag class and lemma as features.
20. MFSFeatures: Most Frequent sense feature.
21. SuperSenseFeatures: Ciaramita and Altun (2006) features for super sense tagging.

Version:

2015-03-27

Author:

ragerri

Constructor Summary

Constructors

Constructor and Description
FixedTrainer(opennlp.tools.util.TrainingParameters params) Construct a trainer.

Method Summary

Modifier and Type	Method and Description
void	createTrainer(opennlp.tools.util.TrainingParameters params) Create TokenNameFinderFactory with custom features.
static void	loadResource(String serializerId, Map<String,opennlp.tools.util.model.ArtifactSerializer> artifactSerializers, String resourcePath, byte[] featureGenDescriptor, Map<String,Object> resources) Load a resource by resourceId.
static Map<String,Object>	loadResources(opennlp.tools.util.TrainingParameters params, byte[] featureGenDescriptor) Load the external resources such as gazetters and clustering lexicons.

Methods inherited from class eus.ixa.ixa.pipe.nerc.train.AbstractTrainer

getBeamSize, getLanguage, getNameClassifierFactory, getNameStream, getSequenceCodec, setLanguage, setNameClassifierFactory, setSequenceCodec, train

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FixedTrainer

public FixedTrainer(opennlp.tools.util.TrainingParameters params)
             throws IOException

Construct a trainer.

Parameters:

params - the training parameters

Throws:

IOException - if io errors

Method Detail

createTrainer

public void createTrainer(opennlp.tools.util.TrainingParameters params)
                   throws IOException

Create TokenNameFinderFactory with custom features.

Parameters:

params - the parameter training file

Throws:

IOException - if io error

loadResources

public static Map<String,Object> loadResources(opennlp.tools.util.TrainingParameters params,
                                               byte[] featureGenDescriptor)
                                        throws IOException

Load the external resources such as gazetters and clustering lexicons.

Parameters:

params - the training parameters

featureGenDescriptor - the feature generator descriptor

Returns:

the map contanining and id and the resource

Throws:

IOException - if io error

loadResource

public static void loadResource(String serializerId,
                                Map<String,opennlp.tools.util.model.ArtifactSerializer> artifactSerializers,
                                String resourcePath,
                                byte[] featureGenDescriptor,
                                Map<String,Object> resources)

Load a resource by resourceId.

Parameters:

serializerId - the serializer id

artifactSerializers - the serializers in which to put the resource

resourcePath - the canonical path of the resource

featureGenDescriptor - the feature descriptor

resources - the map in which to put the resource