| Package | Description |
|---|---|
| org.cleartk.token.tokenizer |
| Modifier and Type | Method and Description |
|---|---|
List<Token> |
Tokenizer_ImplBase.getTokens(String text)
performs tokenization on the input text and returns the character offsets corresponding to the
begining and end of each token w.r.t.
|
List<Token> |
Tokenizer.getTokens(String text) |
Copyright © 2014. All rights reserved.