se.lth.cs.srl.preprocessor.tokenization

Class StanfordChineseSegmenterWrapper

java.lang.Object

se.lth.cs.srl.preprocessor.tokenization.StanfordChineseSegmenterWrapper

All Implemented Interfaces:

Tokenizer

public class StanfordChineseSegmenterWrapper
extends Object
implements Tokenizer

This is a wrapper for the Stanford Chinese Segmenter, version 3.2.0

Author:

anders bjorkelund

Constructor Summary

Constructors

Constructor and Description
StanfordChineseSegmenterWrapper(File dataDir) Initialize the segmenter

Method Summary

Modifier and Type	Method and Description
static void	main(String[] args)
String[]	tokenize(String sentence) Tokenize a sentence.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StanfordChineseSegmenterWrapper

public StanfordChineseSegmenterWrapper(File dataDir)

Initialize the segmenter

Parameters:

dataDir - this is the 'datadir' from the 2008-05-21 distribution.

Method Detail

tokenize

public String[] tokenize(String sentence)

Description copied from interface: Tokenizer

Tokenize a sentence. The returned array contains a root-token

Specified by:

tokenize in interface Tokenizer

Parameters:

sentence - The sentence to tokenize

Returns:

a root token, followed by the forms

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception