edu.nyu.jet.format

Class PTBReader

java.lang.Object

edu.nyu.jet.format.PTBReader

public class PTBReader
extends Object

A reader for the output of a Penn Treebank Parser. The methods read a Penn Treebank corpus and either annotate an existing Document (addAnnotations methods) with constit annotations representing the trees or build a new Jet.Tipster.Document from the parse trees.

Author:

Akira ODA

Constructor Summary

Constructors

Constructor and Description
PTBReader()

Method Summary

Methods

Modifier and Type	Method and Description
void	addAnnotations(List<ParseTreeNode> trees, Document doc, String targetAnnotation, Span span, boolean jetCategories) Adds constit annotations to an existing Document doc to represent the parse tree structure of a set of trees trees.
void	addAnnotations(List<ParseTreeNode> trees, List<Integer> offsets, Document doc, String targetAnnotation, Span span, boolean jetCategories) Adds constit annotations to an existing Document doc to represent the parse tree structure of a set of trees trees.
void	addAnnotations(ParseTreeNode tree, Document doc, Span span, boolean jetCategories) Adds constit annotations to an existing Document doc to represent the parse tree structure tree.
List<Integer>	getOffsets()
Treebank	load(File file) Builds Document object from Penn treebank corpus.
Treebank	load(File file, String encoding) Builds Document object from Penn treebank corpus.
Treebank	load(Reader in) Builds Jet.Tipster.Document object from Penn treebank corpus.
List<ParseTreeNode>	loadParseTrees(File file)
List<ParseTreeNode>	loadParseTrees(Reader in) Loads parse tree corpus from Penn Treebank corpus.
static void	main(String[] args) converts a set of Penn TreeBank files into text documents.
void	setAddingToken(boolean b) Sets a adding tokens automatically or not.
void	setBackslashAsEscapeCharacter(boolean b) Sets a backslash is treated as escape character or not.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PTBReader

public PTBReader()

Method Detail

addAnnotations

public void addAnnotations(ParseTreeNode tree,
                  Document doc,
                  Span span,
                  boolean jetCategories)

Adds constit annotations to an existing Document doc to represent the parse tree structure tree.

Parameters:

tree - the parse tree (for a portion of Document doc)

doc - the document

span - the portion of doc covered by the parse tree

jetCategories - if true, use Jet categories as terminal categories (if false, use categories read from parse trees)

addAnnotations

public void addAnnotations(List<ParseTreeNode> trees,
                  Document doc,
                  String targetAnnotation,
                  Span span,
                  boolean jetCategories)

Adds constit annotations to an existing Document doc to represent the parse tree structure of a set of trees trees.

Parameters:

trees - list of parse trees

doc - document to which annotations should be added

targetAnnotation - name of annotation to determine spans to add parse tree annotations.

span - target span.

jetCategories - if false, use lexical categories from Penn Tree Bank; if true, use categories from Jet

addAnnotations

public void addAnnotations(List<ParseTreeNode> trees,
                  List<Integer> offsets,
                  Document doc,
                  String targetAnnotation,
                  Span span,
                  boolean jetCategories)

Adds constit annotations to an existing Document doc to represent the parse tree structure of a set of trees trees. This version is provided for parse tree files which include sentence offsets.

Parameters:

trees - list of parse trees

offsets - list of the starting position (in doc) of the text corresponding to each parse tree

doc - document to which annotations should be added

targetAnnotation - name of annotation to get 'parse' feature pointing to parse tree

span - target span.

jetCategories - if false, use lexical categories from Penn Tree Bank; if true, use categories from Jet

loadParseTrees

public List<ParseTreeNode> loadParseTrees(Reader in)
                                   throws IOException,
                                          InvalidFormatException

Loads parse tree corpus from Penn Treebank corpus.

This method loads the parse trees, but not determine annotation span and not set annotation.

Also sets offsets to a list of the sentence offsets, if they are encoded as comments preceding each tree.

Parameters:

in - the Reader from which the Penn Trees are read

Returns:

a List of parse trees

Throws:

IOException

InvalidFormatException

loadParseTrees

public List<ParseTreeNode> loadParseTrees(File file)
                                   throws IOException,
                                          InvalidFormatException

Throws:

IOException

InvalidFormatException

getOffsets

public List<Integer> getOffsets()

load

public Treebank load(Reader in)
              throws IOException,
                     InvalidFormatException

Builds Jet.Tipster.Document object from Penn treebank corpus.

Parameters:

in -

Returns:

Throws:

IOException

InvalidFormatException

load

public Treebank load(File file)
              throws IOException,
                     InvalidFormatException

Builds Document object from Penn treebank corpus.

Parameters:

file -

Returns:

Throws:

IOException

InvalidFormatException

load

public Treebank load(File file,
            String encoding)
              throws IOException,
                     InvalidFormatException

Builds Document object from Penn treebank corpus.

Parameters:

file -

encoding -

Returns:

Throws:

IOException

InvalidFormatException

setBackslashAsEscapeCharacter

public void setBackslashAsEscapeCharacter(boolean b)

Sets a backslash is treated as escape character or not.

Parameters:

b -

setAddingToken

public void setAddingToken(boolean b)

Sets a adding tokens automatically or not.

Parameters:

b -

main

public static void main(String[] args)
                 throws Exception

converts a set of Penn TreeBank files into text documents. Invoked by: PTBReader inputDir outputDir. Converts all files with extension .mrg in inputDir to text documents, and writes them into outputDir.

Throws:

Exception