se.lth.cs.srl.preprocessor.tokenization

Class StanfordFrenchTokenizer

java.lang.Object

se.lth.cs.srl.preprocessor.tokenization.StanfordFrenchTokenizer

All Implemented Interfaces:

Tokenizer

public class StanfordFrenchTokenizer
extends Object
implements Tokenizer

Constructor Summary

Constructors

Constructor and Description
StanfordFrenchTokenizer()

Method Summary

Modifier and Type	Method and Description
static void	main(String[] args)
String[]	tokenize(String sentence) Tokenize a sentence.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StanfordFrenchTokenizer

public StanfordFrenchTokenizer()

Method Detail

tokenize

public String[] tokenize(String sentence)

Description copied from interface: Tokenizer

Tokenize a sentence. The returned array contains a root-token

Specified by:

tokenize in interface Tokenizer

Parameters:

sentence - The sentence to tokenize

Returns:

a root token, followed by the forms

main

public static void main(String[] args)