edu.nyu.jet.parser

Class StatParser

java.lang.Object

edu.nyu.jet.parser.StatParser

public class StatParser
extends Object

an interface to Dan Bikel's implementation of Collins' parser

Constructor Summary

Constructors

Constructor and Description
StatParser()

Method Summary

Methods

Modifier and Type	Method and Description
static void	buildParserInput(Document doc, int start, int end, boolean setPOS) build the arrays 'words', 'spans', 'wordDefns', and 'pennPOS' for the parser: words[i] = the i-th word string, for the PTB parser (normalized to PTB form, such as -LRB- for '(' spans[i] = the span of the i-th word string wordDefns[i] = the Jet word defn of the i-th sentence element (if there are several defns, takes the first one) pennPOS[i] = the PTB POS for the i-th sentence element
static Annotation	buildWordDefn(Document doc, String word, Span span, Annotation wordDefn, String pennPOS)
static void	clearInputAnnotations(Document doc) for ACE: erase all the characters within ANNOTATION ...
static void	deleteUnusedConstits(Document doc, Span span, Annotation rootAnnotation) deletes all annotations of type 'constit' within span 'span' of Document 'doc' which are not descendants of 'rootAnnotation'.
static Set<Annotation>	descendants(Annotation node) returns a Set containing the parse tree node and all of its descendants (its children, the children of its children, etc.).
static void	fixHyphenatedItems(Document doc) for hyphenated forms X-Y, which are treated as three separate tokens by the ACE tokenizer, create a single constituent with category 'hyphword' and Penn POS JJ.
static void	initialize(String dataPath, Properties config) initialize the parser (load grammar and data files).
static void	initialize(String propertiesFile, String grammarFile) initialize the parser.
static boolean	isInitialized() return true if the parser has already been initialized.
static void	main(String[] args) Apply the Bikel parser to a collection of documents, writing out the resulting parses as XML annotations on the documents.
static ParseTreeNode	parse(Document doc, Span span) parse the sentence in 'span' of Document 'doc'.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StatParser

public StatParser()

Method Detail

main

public static void main(String[] args)
                 throws IOException

Apply the Bikel parser to a collection of documents, writing out the resulting parses as XML annotations on the documents. Takes 4 arguments:

• properties: Jet properties file
• inputDir: directory containing documents to be parsed
• outputDir: directory to contain parsed documents
• fileList: list of document file names

If a parsed document file is already present in outputDir, it is not overwritten; processing of this document is skipped. This allows processing to be easily interrupted and restarted. On the other hand, it means that if you want documents to be reparsed, the old parsed documents must be deleted.

Throws:

IOException

initialize

public static void initialize(String dataPath,
              Properties config)

initialize the parser (load grammar and data files). File names are obtainted from properties 'StatParser.properties.fileName' and 'StatParser.grammar.fileName', and are relative to 'dataPath'.

initialize

public static void initialize(String propertiesFile,
              String grammarFile)

initialize the parser. Load the properties from 'propertiesFile' and the grammar from file 'grammarFile'.

isInitialized

public static boolean isInitialized()

return true if the parser has already been initialized.

parse

public static ParseTreeNode parse(Document doc,
                  Span span)

parse the sentence in 'span' of Document 'doc'. The sentence must have been tokenized and tagged with Penn tags. The parse is returned as a nested set of ParseTreeNodes and is also added as 'constit' annotations on the Document.

deleteUnusedConstits

public static void deleteUnusedConstits(Document doc,
                        Span span,
                        Annotation rootAnnotation)

deletes all annotations of type 'constit' within span 'span' of Document 'doc' which are not descendants of 'rootAnnotation'. This method is used to delete unused word definitions after a parse tree has been built.

buildParserInput

public static void buildParserInput(Document doc,
                    int start,
                    int end,
                    boolean setPOS)

build the arrays 'words', 'spans', 'wordDefns', and 'pennPOS' for the parser: words[i] = the i-th word string, for the PTB parser (normalized to PTB form, such as -LRB- for '(' spans[i] = the span of the i-th word string wordDefns[i] = the Jet word defn of the i-th sentence element (if there are several defns, takes the first one) pennPOS[i] = the PTB POS for the i-th sentence element

fixHyphenatedItems

public static void fixHyphenatedItems(Document doc)

for hyphenated forms X-Y, which are treated as three separate tokens by the ACE tokenizer, create a single constituent with category 'hyphword' and Penn POS JJ.

buildWordDefn

public static Annotation buildWordDefn(Document doc,
                       String word,
                       Span span,
                       Annotation wordDefn,
                       String pennPOS)

descendants

public static Set<Annotation> descendants(Annotation node)

returns a Set containing the parse tree node and all of its descendants (its children, the children of its children, etc.).

Parameters:

node - an Annotation representing a parse tree node (an Annotation of type 'constit').

clearInputAnnotations

public static void clearInputAnnotations(Document doc)

for ACE: erase all the characters within ANNOTATION ... /ANNOTATION