edu.nyu.jet.hmm

Class HMMNameTagger

java.lang.Object

edu.nyu.jet.hmm.HMMNameTagger

All Implemented Interfaces:

NameTagger

public class HMMNameTagger
extends Object
implements NameTagger

a Named Entity tagger based on the generic HMM (Hidden Markov Model) mechanism. Methods are provided for creating an HMM for a set of name tags, for training the HMM from annotated corpora, for applying the tagger to new text, and for scoring the results. It uses an external file consisting of a tag table, the line 'endtags', and the HMM.

Field Summary

Fields

Modifier and Type	Field and Description
HMMannotator	annotator
HMM	nameHMM

Constructor Summary

Constructors

Constructor and Description
HMMNameTagger(Class emitterClass) creates a new HMMNameTagger (with an empty HMM).

Method Summary

Methods

Modifier and Type	Method and Description
void	buildNameHMM(String tagFileName) create the HMM states and arcs.
static boolean	inZone(Document doc, Span span, String zoneType) returns 'true' if Span 'span' is enclosed in an annotation of type 'zoneType'.
void	load(String fileName) load the tag table and the HMM associated with this tagger from file 'fileName'.
static void	main(String[] args) procedure the named entity tagger on documents with in-line NE tags.
void	newDocument()
void	store(String fileName) store the tag table and the HMM associated with this tagger to file 'fileName'.
void	tag(Document doc, Span span) tag span 'span' of Document 'doc' with Named Entity annotations.
void	tagDocument(Document doc)
static void	tagPersonZone(Document doc, Span span, HMMannotator annotator)
void	train(ExternalDocument doc)
void	train(String trainingCollection) train the HMMNameTagger using the collection of Documents 'trainingCollection'.
void	train(String directory, String fileList)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

nameHMM

public HMM nameHMM

annotator

public HMMannotator annotator

Constructor Detail

HMMNameTagger

public HMMNameTagger(Class emitterClass)

creates a new HMMNameTagger (with an empty HMM).

Parameters:

emitterClass - the class of the emitter associated with each state of the HMM; must be a subclass of HMMemitter.

Method Detail

buildNameHMM

public void buildNameHMM(String tagFileName)

create the HMM states and arcs. This is the first step in building the HMM, prior to training. The list of annotation types and features is taken from file 'tagFileName'.

Creates six states for each name type.

train

public void train(String trainingCollection)
           throws IOException

train the HMMNameTagger using the collection of Documents 'trainingCollection'. The documents should have a TEXT zone marked; training is done on all sentences within this zone.

Throws:

IOException

train

public void train(String directory,
         String fileList)
           throws IOException

Throws:

IOException

train

public void train(ExternalDocument doc)
           throws IOException

Throws:

IOException

store

public void store(String fileName)
           throws IOException

store the tag table and the HMM associated with this tagger to file 'fileName'.

Throws:

IOException

load

public void load(String fileName)
          throws IOException

load the tag table and the HMM associated with this tagger from file 'fileName'.

Specified by:

load in interface NameTagger

Throws:

IOException

tagDocument

public void tagDocument(Document doc)

Specified by:

tagDocument in interface NameTagger

newDocument

public void newDocument()

Specified by:

newDocument in interface NameTagger

tag

public void tag(Document doc,
       Span span)

tag span 'span' of Document 'doc' with Named Entity annotations.

Specified by:

tag in interface NameTagger

inZone

public static boolean inZone(Document doc,
             Span span,
             String zoneType)

returns 'true' if Span 'span' is enclosed in an annotation of type 'zoneType'. This function is used to detect special markup in the text, such as 'SPEAKER', which indicates a name of a particular type.

tagPersonZone

public static void tagPersonZone(Document doc,
                 Span span,
                 HMMannotator annotator)

main

public static void main(String[] args)
                 throws IOException

procedure the named entity tagger on documents with in-line NE tags. Takes 3+2n arguments:

• state-file: file defining states of the HMM (input)
• model-file: file containing trained HMM model (output)
• unigram|bigram: should model use unigram or bigram probabilities
• directory-1: directory containing ne-tagged files
• filelist-1: list of ne-tagged files
• directory-2
• filelist-2
• ...

Throws:

IOException