org.maochen.nlp.ml.util

Class TrainingDataUtils

java.lang.Object

org.maochen.nlp.ml.util.TrainingDataUtils

public class TrainingDataUtils
extends Object

Created by Maochen on 8/10/15.

Constructor Summary

Constructors

Constructor and Description
TrainingDataUtils()

Method Summary

Modifier and Type	Method and Description
static List<org.maochen.nlp.ml.Tuple>	createBalancedTrainingData(List<org.maochen.nlp.ml.Tuple> trainingData)
static List<org.maochen.nlp.ml.SequenceTuple>	readSeqFile(InputStream trainingFile, String delimiter, int tagCol)
static org.apache.commons.lang3.tuple.Pair<List<org.maochen.nlp.ml.Tuple>,List<org.maochen.nlp.ml.Tuple>>	splitData(List<org.maochen.nlp.ml.Tuple> trainingData, double proportion) Shuffle the data and split by proportion

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TrainingDataUtils

public TrainingDataUtils()

Method Detail

createBalancedTrainingData

public static List<org.maochen.nlp.ml.Tuple> createBalancedTrainingData(List<org.maochen.nlp.ml.Tuple> trainingData)

splitData

public static org.apache.commons.lang3.tuple.Pair<List<org.maochen.nlp.ml.Tuple>,List<org.maochen.nlp.ml.Tuple>> splitData(List<org.maochen.nlp.ml.Tuple> trainingData,
                                                                                                                           double proportion)

Shuffle the data and split by proportion

Parameters:

trainingData - whole data collection.

proportion - scale from 0.1 - 1.0.

Returns:

Left- small chunk. Right - large chunk.

readSeqFile

public static List<org.maochen.nlp.ml.SequenceTuple> readSeqFile(InputStream trainingFile,
                                                                 String delimiter,
                                                                 int tagCol)