org.openkoreantext.processor

Class OpenKoreanTextProcessorJava

java.lang.Object

org.openkoreantext.processor.OpenKoreanTextProcessorJava

public final class OpenKoreanTextProcessorJava
extends Object

Java wrapper for TwitterKoreanProcessor using Builder pattern

Constructor Summary

Constructors

Constructor and Description
OpenKoreanTextProcessorJava()

Method Summary

Modifier and Type	Method and Description
static void	addNounsToDictionary(List<String> words) Add user-defined words to the noun dictionary.
static String	detokenize(List<String> tokens) Detokenize the input list of words.
static List<KoreanPhraseExtractor.KoreanPhrase>	extractPhrases(scala.collection.Seq<KoreanToken> tokens, boolean filterSpam, boolean includeHashtags) Extract phrases from Korean input text
static CharSequence	normalize(CharSequence text) Normalize Korean text 그랰ㅋㅋㅋㅋㅋㅋ -> 그래ㅋㅋ
static List<Sentence>	splitSentences(CharSequence text) Split input text into sentences.
static scala.collection.Seq<KoreanToken>	tokenize(CharSequence text) Tokenize with the builder options.
static List<KoreanTokenJava>	tokensToJavaKoreanTokenList(scala.collection.Seq<KoreanToken> tokens)
static List<KoreanTokenJava>	tokensToJavaKoreanTokenList(scala.collection.Seq<KoreanToken> tokens, boolean keepSpace) Transforms the tokenization output to List
static List<String>	tokensToJavaStringList(scala.collection.Seq<KoreanToken> tokens)
static List<String>	tokensToJavaStringList(scala.collection.Seq<KoreanToken> tokens, boolean keepSpace) Tokenize with the builder options into a String Iterable.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OpenKoreanTextProcessorJava

public OpenKoreanTextProcessorJava()

Method Detail

normalize

public static CharSequence normalize(CharSequence text)

Normalize Korean text 그랰ㅋㅋㅋㅋㅋㅋ -> 그래ㅋㅋ

Parameters:

text - Input text.

Returns:

Normalized text.

tokenize

public static scala.collection.Seq<KoreanToken> tokenize(CharSequence text)

Tokenize with the builder options.

Parameters:

text - Input text.

Returns:

A list of Korean Tokens (run tokensToJavaList to transform to Java List)

addNounsToDictionary

public static void addNounsToDictionary(List<String> words)

Add user-defined words to the noun dictionary. Spaced words are ignored.

Parameters:

words - List of user nouns.

tokensToJavaKoreanTokenList

public static List<KoreanTokenJava> tokensToJavaKoreanTokenList(scala.collection.Seq<KoreanToken> tokens,
                                                                boolean keepSpace)

Transforms the tokenization output to List

Parameters:

tokens - Korean tokens (output of tokenize(CharSequence text)).

Returns:

List of KoreanTokenJava.

tokensToJavaKoreanTokenList

public static List<KoreanTokenJava> tokensToJavaKoreanTokenList(scala.collection.Seq<KoreanToken> tokens)

tokensToJavaStringList

public static List<String> tokensToJavaStringList(scala.collection.Seq<KoreanToken> tokens,
                                                  boolean keepSpace)

Tokenize with the builder options into a String Iterable.

Parameters:

tokens - Korean tokens (output of tokenize(CharSequence text)).

Returns:

List of token strings.

tokensToJavaStringList

public static List<String> tokensToJavaStringList(scala.collection.Seq<KoreanToken> tokens)

splitSentences

public static List<Sentence> splitSentences(CharSequence text)

Split input text into sentences.

Parameters:

text - Input text.

Returns:

List of Sentence objects.

extractPhrases

public static List<KoreanPhraseExtractor.KoreanPhrase> extractPhrases(scala.collection.Seq<KoreanToken> tokens,
                                                                      boolean filterSpam,
                                                                      boolean includeHashtags)

Extract phrases from Korean input text

Parameters:

tokens - Korean tokens (output of tokenize(CharSequence text)).

Returns:

List of phrase CharSequences.

detokenize

public static String detokenize(List<String> tokens)

Detokenize the input list of words.

Parameters:

tokens - List of words.

Returns:

Detokenized string.