StringUtils (ixa-pipe-ml 0.0.2 API)

java.lang.Object
- eus.ixa.ixa.pipe.ml.utils.StringUtils

```
public final class StringUtils
extends Object
```
Pattern matching and other utility string functions.

Version:

2013-03-19

Author:

ragerri

Field Summary

Fields
Modifier and Type Field and Description

static Pattern doubleBar
Pattern to remove double bars from disjunct regex.

Fields
Modifier and Type	Field and Description
`static Pattern`	`doubleBar` Pattern to remove double bars from disjunct regex.

Method Summary

All Methods Static Methods Concrete Methods
Modifier and Type	Method and Description
`static void`	`computeShortestEditScript(String wordForm, String lemma, int[][] distance, StringBuffer permutations)` Computes the Shortest Edit Script (SES) to convert a word into its lemma.
`static String[]`	`convertListTokenToArrayStrings(List<Token> tokenizedSentence)` Convert a list of token objects (e.g.
`static String`	`createDisjunctRegexFromList(List<String> words)`
`static String[]`	`decodeLemmas(String[] tokens, Span[] preds)` Decodes the lemma from the word and the induced lemma class.
`static void`	`decodeLemmasToSpans(String[] tokens, Span[] preds)` Decodes the lemma induced type into the lemma and sets it as value of the Span type.
`static String`	`decodeShortestEditScript(String wordForm, String permutations)` Read predicted SES by the lemmatizer model and apply the permutations to obtain the lemma from the wordForm.
`static List<Integer>`	`exactStringFinder(String pattern, String sentence)` Finds a pattern (typically a named entity string) in a sentence string.
`static List<Integer>`	`exactTokenFinder(String pattern, String[] tokens)` Finds a pattern (typically a named entity string) in a tokenized sentence.
`static List<Integer>`	`exactTokenFinderIgnoreCase(String pattern, String[] tokens)` Finds a pattern (typically a named entity string) in a tokenized sentence.
`static List<File>`	`getFilesInDir(File inputPath)` Recursively get every file in a directory and add them to a list.
`static String`	`getSetStringFromList(List<String> posLemmaValues)`
`static String`	`getShortestEditScript(String wordForm, String lemma)` Get the SES required to go from a word to a lemma.
`static String`	`getStringFromTokens(String[] tokens)` Gets the String joined by a space of an array of tokens.
`static int[][]`	`levenshteinDistance(String wordForm, String lemma)` Computes the Levenshtein distance of two strings in a matrix.
`static void`	`splitLine(String line, char delimiter, String[] splitted)` Fast line splitting with a separator, typically a tab or space character.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - doubleBar
```
public static Pattern doubleBar
```
    Pattern to remove double bars from disjunct regex.
- Method Detail
  - exactTokenFinderIgnoreCase
```
public static List<Integer> exactTokenFinderIgnoreCase(String pattern,
                                                       String[] tokens)
```
    Finds a pattern (typically a named entity string) in a tokenized sentence. It outputs the Span indexes of the named entity found, if any.
    
    Parameters:
    
    pattern - a string to find
    
    tokens - an array of tokens
    
    Returns:
    
    token spans of the pattern (e.g. a named entity)
  - exactTokenFinder
```
public static List<Integer> exactTokenFinder(String pattern,
                                             String[] tokens)
```
    Finds a pattern (typically a named entity string) in a tokenized sentence. It outputs the Span indexes of the named entity found, if any
    
    Parameters:
    
    pattern - a string to find
    
    tokens - an array of tokens
    
    Returns:
    
    token spans of the pattern (e.g. a named entity)
  - exactStringFinder
```
public static List<Integer> exactStringFinder(String pattern,
                                              String sentence)
```
    Finds a pattern (typically a named entity string) in a sentence string. It outputs the offsets for the start and end characters named entity found, if any.
    
    Parameters:
    
    pattern - the pattern to be searched
    
    sentence - the sentence
    
    Returns:
    
    a list of integers corresponding to the characters of the string found
  - getSetStringFromList
```
public static String getSetStringFromList(List<String> posLemmaValues)
```
  - getStringFromTokens
```
public static String getStringFromTokens(String[] tokens)
```
    Gets the String joined by a space of an array of tokens.
    
    Parameters:
    
    tokens - an array of tokens representing a tokenized sentence
    
    Returns:
    
    sentence the sentence corresponding to the tokens
  - getFilesInDir
```
public static List<File> getFilesInDir(File inputPath)
```
    Recursively get every file in a directory and add them to a list.
    
    Parameters:
    
    inputPath - the input directory
    
    Returns:
    
    the list containing all the files
  - levenshteinDistance
```
public static int[][] levenshteinDistance(String wordForm,
                                          String lemma)
```
    Computes the Levenshtein distance of two strings in a matrix. Based on pseudo-code provided here: https://en.wikipedia.org/wiki/Levenshtein_distance#Computing_Levenshtein_distance which in turn is based on the paper Wagner, Robert A.; Fischer, Michael J. (1974), "The String-to-String Correction Problem", Journal of the ACM 21 (1): 168-173
    
    Parameters:
    
    wordForm - the form
    
    lemma - the lemma
    
    Returns:
    
    the distance
  - computeShortestEditScript
```
public static void computeShortestEditScript(String wordForm,
                                             String lemma,
                                             int[][] distance,
                                             StringBuffer permutations)
```
    Computes the Shortest Edit Script (SES) to convert a word into its lemma. This is based on Chrupala's PhD thesis (2007).
    
    Parameters:
    
    wordForm - the token
    
    lemma - the target lemma
    
    distance - the levenshtein distance
    
    permutations - the number of permutations
  - decodeShortestEditScript
```
public static String decodeShortestEditScript(String wordForm,
                                              String permutations)
```
    Read predicted SES by the lemmatizer model and apply the permutations to obtain the lemma from the wordForm.
    
    Parameters:
    
    wordForm - the wordForm
    
    permutations - the permutations predicted by the lemmatizer model
    
    Returns:
    
    the lemma
  - getShortestEditScript
```
public static String getShortestEditScript(String wordForm,
                                           String lemma)
```
    Get the SES required to go from a word to a lemma.
    
    Parameters:
    
    wordForm - the word
    
    lemma - the lemma
    
    Returns:
    
    the shortest edit script
  - decodeLemmas
```
public static String[] decodeLemmas(String[] tokens,
                                    Span[] preds)
```
    Decodes the lemma from the word and the induced lemma class.
    
    Parameters:
    
    tokens - the array of tokens
    
    preds - the predicted lemma classes
    
    Returns:
    
    the array of decoded lemmas
  - decodeLemmasToSpans
```
public static void decodeLemmasToSpans(String[] tokens,
                                       Span[] preds)
```
    Decodes the lemma induced type into the lemma and sets it as value of the Span type.
    
    Parameters:
    
    tokens - the tokens in the sentence
    
    preds - the predicted spans
  - createDisjunctRegexFromList
```
public static String createDisjunctRegexFromList(List<String> words)
```
  - convertListTokenToArrayStrings
```
public static String[] convertListTokenToArrayStrings(List<Token> tokenizedSentence)
```
    Convert a list of token objects (e.g. a tokenized sentence) into an array containing the token strings.
    
    Parameters:
    
    tokenizedSentence - the list of token objects
    
    Returns:
    
    the array of token strings
  - splitLine
```
public static void splitLine(String line,
                             char delimiter,
                             String[] splitted)
```
    Fast line splitting with a separator, typically a tab or space character.
    
    Parameters:
    
    line - the line to be splitted
    
    delimiter - the delimiter
    
    splitted - the array containing the splitted tokens for each line

Class StringUtils

Field Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

doubleBar

Method Detail

exactTokenFinderIgnoreCase

exactTokenFinder

exactStringFinder

getSetStringFromList

getStringFromTokens

getFilesInDir

levenshteinDistance

computeShortestEditScript

decodeShortestEditScript

getShortestEditScript

decodeLemmas

decodeLemmasToSpans

createDisjunctRegexFromList

convertListTokenToArrayStrings

splitLine