TreebankFormatParser (ClearTK ClearNLP 2.0.0 API)

java.lang.Object
- org.cleartk.util.treebank.TreebankFormatParser

```
public class TreebankFormatParser
extends Object
```
This class was written to be a stand alone parser for the Penn Treebank data. Basically, I need a way to synch up the propbank data with extents of plain text that are labeled. This is not possible to do without parsing the treebank data first. The parse method will parse a single sentence from the treebank data from e.g. wsj/mrg/06/wsj_0656.mrg. I initially looked at the OpenNLP treebank parser but they made a few assumptions about they wanted to keep for the parser that would make it difficult to align with the propbank data. See: http://sourceforge.net/projects/opennlp/forums/forum/9943/topic/1751983 for relevant discussion. I looked at their parsing implementation and tried to modify it. However, I think the code below bears little resemblance to theirs. But there may yet be some snippets taken directly out of that code. The two regular expressions used are very similar.
Copyright (c) 2007-2008, Regents of the University of Colorado
All rights reserved.

Author:

Philip Ogren

Field Summary

Fields
Modifier and Type	Field and Description
`static String`	`cleanUPRegex1`
`static String`	`cleanUPRegex2`
`static String`	`cleanUPRegex3`
`static String`	`cleanUPRegex4`
`static String`	`LEAF_NODE_REGEX` used to identify tokens in Penn Treebank labeled constituents.
`static String`	`TYPE_REGEX` used to identify the type of a consituent in a treebank parse tree.

Constructor Summary

Constructors
Constructor and Description

TreebankFormatParser()

Constructors
Constructor and Description
`TreebankFormatParser()`

Method Summary

Methods
Modifier and Type	Method and Description
`static TreebankNode`	`getLeafNode(String parseFragment)` Uses the leafNodePattern to identify a string as a terminal.
`static String`	`getType(String parseFragment)` Returns the type of a constituent of some fragment of a treebank parse.
`static String`	`inferPlainText(String treebankText)` A treebank parse does not preserve whitespace information.
`static int`	`movePastWhiteSpaceChars(String text, int textOffset)`
`static boolean`	`parensMatch(String contents)`
`static TopTreebankNode`	`parse(String parse)` Create TreebankNode objects corresponding to the given TreeBank format parse, e.g.:
`static TopTreebankNode`	`parse(String parse, String text, int textOffset)` Create TreebankNode objects corresponding to the given TreeBank format parse, e.g.:
`static List<TopTreebankNode>`	`parseDocument(String parse, int textOffset, String text)` This method parses an entire documents worth of treebanked sentences.
`static String`	`prepareString(String parse)` This method was created simply as a way to clean up the parse string for a sentence in the treebank syntax.
`static String[]`	`splitSentences(String mrgContents)` Generally speaking, we expect one treebanked sentence per line.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - cleanUPRegex1
```
public static final String cleanUPRegex1
```
    See Also:
    Constant Field Values
  - cleanUPRegex2
```
public static final String cleanUPRegex2
```
    See Also:
    Constant Field Values
  - cleanUPRegex3
```
public static final String cleanUPRegex3
```
    See Also:
    Constant Field Values
  - cleanUPRegex4
```
public static final String cleanUPRegex4
```
    See Also:
    Constant Field Values
  - LEAF_NODE_REGEX
```
public static final String LEAF_NODE_REGEX
```
    used to identify tokens in Penn Treebank labeled constituents. It will match strings such as:
    - (NNP Community)
    - (: --)
    - (-NONE- *U*)
    See Also:
    Constant Field Values
  - TYPE_REGEX
```
public static final String TYPE_REGEX
```
    used to identify the type of a consituent in a treebank parse tree. It will match strings such as:
    - "NNP" in "(NNP Community)"
    - ":" in "(: --)"
    - "-NONE-" in "(-NONE- *U*)"
    See Also:
    Constant Field Values
- Constructor Detail
  - TreebankFormatParser
```
public TreebankFormatParser()
```
- Method Detail
  - getLeafNode
```
public static TreebankNode getLeafNode(String parseFragment)
```
    Uses the leafNodePattern to identify a string as a terminal. Examples:
    - parseFragment = "(NNP Community)", returns a leaf node
    - parseFragment = "(QP ($ $) (CD 107) (CD million) )", returns null
    Parameters:
    parseFragment - some fragment of a treebank parse.
    
    Returns:
    if the string matches, then a node will be returned. Otherwise, null is returned.
  - getType
```
public static String getType(String parseFragment)
```
    Returns the type of a constituent of some fragment of a treebank parse. Assumes that the first character is a parenthesis. Examples:
    - parseFragment = "(NP-LOC (NNP Calif.) )" return = "NP-LOC"
    - parseFragment = "(NP" return "NP"
    - parseFragment = "(-NONE- *U*) ) (PP (IN of)" return = "-NONE-"
    Parameters:
    parseFragment - some fragment of a treebank parse
    
    Returns:
    the type of the constituent.
  - inferPlainText
```
public static String inferPlainText(String treebankText)
```
    A treebank parse does not preserve whitespace information. This method provides a simple mechanism for inferring the original plain text of a treebank parse. If you have access to the original plain text, then you can bypass use of this method by calling the appropriate parse method.
    
    Parameters:
    treebankText - One or more parses in Treebank parenthesized format.
    
    Returns:
    a "best" guess of the original plain text given in the parse.
    See Also:
    parse(String, String, int)
  - movePastWhiteSpaceChars
```
public static int movePastWhiteSpaceChars(String text,
                          int textOffset)
```
  - parensMatch
```
public static boolean parensMatch(String contents)
```
  - parse
```
public static TopTreebankNode parse(String parse)
```
    Create TreebankNode objects corresponding to the given TreeBank format parse, e.g.:
```
 ( (X (NP (NP (NML (NN Complex ) (NN trait )) (NN analysis )) (PP (IN of ) (NP (DT the ) (NN mouse ) (NN striatum )))) (: : ) (S (NP-SBJ (JJ independent ) (NNS QTLs )) (VP (VBP modulate ) (NP (NP (NN volume )) (CC and ) (NP (NN neuron ) (NN number)))))) )
 
```
    The text will be inferred automatically from the words in the parse.
    Parameters:
    parse - A TreeBank formatted parse
    
    Returns:
    The TreebankNode root of the parse tree
    See Also:
    inferPlainText(String), parse(String, String, int)
  - parse
```
public static TopTreebankNode parse(String parse,
                    String text,
                    int textOffset)
```
    Create TreebankNode objects corresponding to the given TreeBank format parse, e.g.:
```
 ( (X (NP (NP (NML (NN Complex ) (NN trait )) (NN analysis )) (PP (IN of ) (NP (DT the ) (NN mouse ) (NN striatum )))) (: : ) (S (NP-SBJ (JJ independent ) (NNS QTLs )) (VP (VBP modulate ) (NP (NP (NN volume )) (CC and ) (NP (NN neuron ) (NN number)))))) )
 
```
    The start and end offsets of each TreebankNode will be aligned to the word offsets in the given text.
    Parameters:
    parse - A TreeBank formatted parse
    text - The text to which the parse should be aligned
    textOffset - The character offset at which the parse text should start to be aligned. For example, if the words of the parse start right at the beginning of the text, the appropriate textOffset is 0.
    
    Returns:
    The TreebankNode root of the parse tree. The root node will be a TopTreebankNode, and all its descendants will be TreebankNodes.
    See Also:
    TopTreebankNode, TreebankNode
  - parseDocument
```
public static List<TopTreebankNode> parseDocument(String parse,
                                  int textOffset,
                                  String text)
```
    This method parses an entire documents worth of treebanked sentences.
    
    Parameters:
    parse - a single document provided as treebank parenthesized parses
    textOffset - a value that corresponds to the character offset of the first character of the document. The appropriate value for this method will typically be 0.
    text - a single document provided as plain text. If you do not have access to the original plain text of the document, you can generate some using inferPlainText(String).
  - prepareString
```
public static String prepareString(String parse)
```
    This method was created simply as a way to clean up the parse string for a sentence in the treebank syntax. The most important thing that it does is add a type called TOP to the top node of the sentence. This simplifies parsing. The other string replacements just remove white space and such and are probably unnecessary. This was inspired by the OpenNLP solution which takes in one line at a time from a file that has been modified in this way.
    
    Parameters:
    parse - a String in the treebank format
    
    Returns:
    a String in the treebank that has been cleaned up a bit.
  - splitSentences
```
public static String[] splitSentences(String mrgContents)
```
    Generally speaking, we expect one treebanked sentence per line. This method will simply return the lines of a document assuming that each line has matching parentheses. However, the native penn treebank data contains parsed sentences that are broken up across multiple lines. Each sentence in the PTB starts with "( (S..." and so we split on this to get the sentences. If this method sees "( (S...", then it will return the contents split on that pattern. If not, it will return the lines of the input string. Splits an .mrg file (e.g. wsj/mrg/00/wsj_0020.mrg) into sentence parses.
    
    Returns:
    individual sentence parses from treebank - i.e. strings of the form "( (S..."

Class TreebankFormatParser

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

cleanUPRegex1

cleanUPRegex2

cleanUPRegex3

cleanUPRegex4

LEAF_NODE_REGEX

TYPE_REGEX

Constructor Detail

TreebankFormatParser

Method Detail

getLeafNode

getType

inferPlainText

movePastWhiteSpaceChars

parensMatch

parse

parse

parseDocument

prepareString

splitSentences