se.lth.cs.srl.preprocessor.tokenization

Class OpenNLPToolsTokenizerWrapper

java.lang.Object

se.lth.cs.srl.preprocessor.tokenization.OpenNLPToolsTokenizerWrapper

All Implemented Interfaces:

Tokenizer

public class OpenNLPToolsTokenizerWrapper
extends Object
implements Tokenizer

Constructor Summary

Constructors

Constructor and Description
OpenNLPToolsTokenizerWrapper(opennlp.tools.tokenize.Tokenizer tokenizerImplementation)

Method Summary

Modifier and Type	Method and Description
static OpenNLPToolsTokenizerWrapper	loadOpenNLPTokenizer(File modelFile)
String[]	tokenize(String sentence) Tokenize a sentence.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OpenNLPToolsTokenizerWrapper

public OpenNLPToolsTokenizerWrapper(opennlp.tools.tokenize.Tokenizer tokenizerImplementation)

Method Detail

tokenize

public String[] tokenize(String sentence)

Description copied from interface: Tokenizer

Tokenize a sentence. The returned array contains a root-token

Specified by:

tokenize in interface Tokenizer

Parameters:

sentence - The sentence to tokenize

Returns:

a root token, followed by the forms

loadOpenNLPTokenizer

public static OpenNLPToolsTokenizerWrapper loadOpenNLPTokenizer(File modelFile)
                                                         throws IOException

Throws:

IOException