Uses of Interface
se.lth.cs.srl.preprocessor.tokenization.Tokenizer

Packages that use Tokenizer

Package	Description
se.lth.cs.srl.languages
se.lth.cs.srl.preprocessor
se.lth.cs.srl.preprocessor.tokenization

Uses of Tokenizer in se.lth.cs.srl.languages

Methods in se.lth.cs.srl.languages that return Tokenizer

Modifier and Type	Method and Description
Tokenizer	Language.getTokenizer(File tokenModelFile)

Uses of Tokenizer in se.lth.cs.srl.preprocessor

Fields in se.lth.cs.srl.preprocessor declared as Tokenizer

Modifier and Type	Field and Description
protected Tokenizer	Preprocessor.tokenizer

Constructors in se.lth.cs.srl.preprocessor with parameters of type Tokenizer

Constructor and Description
Preprocessor(Tokenizer tokenizer, is2.lemmatizer.Lemmatizer lemmatizer, is2.tag.Tagger tagger, is2.mtag.Tagger mtagger, is2.parser.Parser parser)

Uses of Tokenizer in se.lth.cs.srl.preprocessor.tokenization

Classes in se.lth.cs.srl.preprocessor.tokenization that implement Tokenizer

Modifier and Type	Class and Description
class	ExnerSwedishTokenizer
class	OpenNLPToolsTokenizerWrapper
class	StanfordChineseSegmenterWrapper This is a wrapper for the Stanford Chinese Segmenter, version 3.2.0
class	StanfordFrenchTokenizer
class	StanfordPTBTokenizer
class	WhiteSpaceTokenizer