se.lth.cs.srl.preprocessor.tokenization

Class WhiteSpaceTokenizer

java.lang.Object

se.lth.cs.srl.preprocessor.tokenization.WhiteSpaceTokenizer

All Implemented Interfaces:

Tokenizer

public class WhiteSpaceTokenizer
extends Object
implements Tokenizer

Constructor Summary

Constructors

Constructor and Description
WhiteSpaceTokenizer()

Method Summary

Modifier and Type	Method and Description
static void	main(String[] args)
String[]	tokenize(String sentence) Tokenize a sentence.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WhiteSpaceTokenizer

public WhiteSpaceTokenizer()

Method Detail

tokenize

public String[] tokenize(String sentence)

Description copied from interface: Tokenizer

Tokenize a sentence. The returned array contains a root-token

Specified by:

tokenize in interface Tokenizer

Parameters:

sentence - The sentence to tokenize

Returns:

a root token, followed by the forms

main

public static void main(String[] args)