eus.ixa.ixa.pipe.pos.train

Class BaselineContextGenerator

java.lang.Object

eus.ixa.ixa.pipe.pos.train.BaselineContextGenerator

All Implemented Interfaces:

opennlp.tools.postag.POSContextGenerator, opennlp.tools.util.BeamSearchContextGenerator<String>

public class BaselineContextGenerator
extends Object
implements opennlp.tools.postag.POSContextGenerator

An improved context generator for the POS Tagger. This baseline generator provides more contextual features such as bigrams to the @DefaultPOSContextGenerator. These extra features require at least 2GB memory to train, more if training data is large.

Version:

2014-07-08

Author:

ragerri

Constructor Summary

Constructors

Constructor and Description
BaselineContextGenerator(opennlp.tools.dictionary.Dictionary aDict) Initializes the current instance.
BaselineContextGenerator(int cacheSize, opennlp.tools.dictionary.Dictionary aDict) Initializes the current instance.

Method Summary

Methods

Modifier and Type	Method and Description
String[]	getContext(int index, Object[] tokens, String[] tags) Returns the context for making a pos tag decision at the specified token index given the specified tokens and previous tags.
String[]	getContext(int index, String[] sequence, String[] priorDecisions, Object[] additionalContext)
protected static String[]	getPrefixes(String lex) Obtain prefixes for each token.
protected static String[]	getSuffixes(String lex) Obtain suffixes for each token.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BaselineContextGenerator

public BaselineContextGenerator(opennlp.tools.dictionary.Dictionary aDict)

Initializes the current instance.

Parameters:

aDict - the dictionary

BaselineContextGenerator

public BaselineContextGenerator(int cacheSize,
                        opennlp.tools.dictionary.Dictionary aDict)

Initializes the current instance.

Parameters:

cacheSize - the cache size

aDict - the dictionary

Method Detail

getPrefixes

protected static String[] getPrefixes(String lex)

Obtain prefixes for each token.

Parameters:

lex - the current word

Returns:

the prefixes

getSuffixes

protected static String[] getSuffixes(String lex)

Obtain suffixes for each token.

Parameters:

lex - the word

Returns:

the suffixes

getContext

public final String[] getContext(int index,
                  String[] sequence,
                  String[] priorDecisions,
                  Object[] additionalContext)

Specified by:

getContext in interface opennlp.tools.postag.POSContextGenerator

Specified by:

getContext in interface opennlp.tools.util.BeamSearchContextGenerator<String>

getContext

public final String[] getContext(int index,
                  Object[] tokens,
                  String[] tags)

Returns the context for making a pos tag decision at the specified token index given the specified tokens and previous tags.

Parameters:

index - The index of the token for which the context is provided.

tokens - The tokens in the sentence.

tags - The tags assigned to the previous words in the sentence.

Returns:

The context for making a pos tag decision at the specified token index given the specified tokens and previous tags.