org.languagetool.tokenizers.es
Class SpanishWordTokenizer

java.lang.Object
  org.languagetool.tokenizers.es.SpanishWordTokenizer

All Implemented Interfaces:

Tokenizer

public class SpanishWordTokenizer
extends Object
implements Tokenizer

Tokenizes a sentence into words. Punctuation and whitespace gets its own token.

Author:

Juan Martorell

Constructor Summary

SpanishWordTokenizer()

Method Summary

List<String>

tokenize(String text)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SpanishWordTokenizer

public SpanishWordTokenizer()

Method Detail

tokenize

public List<String> tokenize(String text)

Specified by:

tokenize in interface Tokenizer