Package eus.ixa.ixa.pipe.ml.tok

Package containing the Sentence Segmenter and Tokenizer classes.

See: Description

Interface Summary

Interface	Description
SentenceSegmenter
Tokenizer

Class Summary

Class	Description
NonPeriodBreaker	This class implements exceptions for periods as sentence breakers and tokens.
Normalizer	Normalizer class for converting punctuation mostly following various corpora conventions such as Penn TreeBank, Ancora, Tutpenn, Tiger and CTAG.
RuleBasedSegmenter	Rule based SentenceSegmenter.
RuleBasedTokenizer	This class provides a multilingual rule based tokenizer.
Token	A Token object contains a single String, a startOffset and the length of the String.
TokenFactory

Package eus.ixa.ixa.pipe.ml.tok Description

Package containing the Sentence Segmenter and Tokenizer classes.

Author:

ragerri