org.languagetool.tokenizers.pt

Class PortugueseWordTokenizer

java.lang.Object

org.languagetool.tokenizers.pt.PortugueseWordTokenizer

All Implemented Interfaces:

Tokenizer

public class PortugueseWordTokenizer
extends Object
implements Tokenizer

Tokenizes a sentence into words. Punctuation and whitespace gets its own token.

Since:

3.6

Author:

Tiago F. Santos based on Spanish tokenizer

Constructor Summary

Constructors

Constructor and Description
PortugueseWordTokenizer()

Method Summary

Modifier and Type	Method and Description
List<String>	tokenize(String text)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PortugueseWordTokenizer

public PortugueseWordTokenizer()

Method Detail

tokenize

public List<String> tokenize(String text)

Specified by:

tokenize in interface Tokenizer