eus.ixa.ixa.pipe.ml.tok

Class TokenFactory

java.lang.Object

eus.ixa.ixa.pipe.ml.tok.TokenFactory

public class TokenFactory
extends Object

Constructor Summary

Constructors

Constructor and Description
TokenFactory() Constructor for a new token factory which will add in the word and the begin/end position annotations.
TokenFactory(boolean addOffsets) Constructor that allows one to choose if index annotation indicating begin/end position will be included in the token.

Method Summary

Modifier and Type	Method and Description
Token	createToken(String tokenString, int offset, int length) Constructs a Token as a String with corresponding offsets and length from which to calculate start and end position of the Token.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TokenFactory

public TokenFactory()

Constructor for a new token factory which will add in the word and the begin/end position annotations.

TokenFactory

public TokenFactory(boolean addOffsets)

Constructor that allows one to choose if index annotation indicating begin/end position will be included in the token.

Parameters:

addOffsets - if true, offSet annotations will be included (this is the default)

Method Detail

createToken

public Token createToken(String tokenString,
                         int offset,
                         int length)

Constructs a Token as a String with corresponding offsets and length from which to calculate start and end position of the Token. (Does not take substring).

Parameters:

tokenString - string to be added to a Token object

offset - the starting offset of the Token

length - of the string

Returns:

a new Token object