ITEM_TYPE - The type of items that define the train and test data in
trainAndTest(List, List) and crossValidation(List, int), and that are
used to create CollectionReaders in getCollectionReader(List). A common
choice for this parameter when working with data on a filesystem is File.STATS_TYPE - The type of statistics object that will be returned by testing methods. A common choice
for this parameter is AnnotationStatistics.public abstract class Evaluation_ImplBase<ITEM_TYPE,STATS_TYPE> extends Object
| Modifier and Type | Field and Description |
|---|---|
protected File |
baseDirectory |
| Constructor and Description |
|---|
Evaluation_ImplBase(File baseDirectory)
Create an evaluation that will write all auxiliary files to the given directory.
|
| Modifier and Type | Method and Description |
|---|---|
List<STATS_TYPE> |
crossValidation(List<ITEM_TYPE> items,
int nFolds)
Run a cross-validation.
|
protected abstract CollectionReader |
getCollectionReader(List<ITEM_TYPE> items)
Creates a
CollectionReader from the given items. |
protected List<ITEM_TYPE> |
selectFoldTestItems(List<ITEM_TYPE> items,
int nFolds,
int fold)
Determines which items should be used for testing in one fold of a cross-validation.
|
protected List<ITEM_TYPE> |
selectFoldTrainItems(List<ITEM_TYPE> items,
int nFolds,
int fold)
Determines which items should be used for training in one fold of a cross-validation.
|
protected abstract STATS_TYPE |
test(CollectionReader collectionReader,
File directory)
Evaluates a model on a set of testing data.
|
protected abstract void |
train(CollectionReader collectionReader,
File directory)
Trains a model on a set of training data.
|
STATS_TYPE |
trainAndTest(List<ITEM_TYPE> trainItems,
List<ITEM_TYPE> testItems)
Train a model on one set of items and test it on another.
|
protected File baseDirectory
public Evaluation_ImplBase(File baseDirectory)
baseDirectory - The directory for all evaluation files.public List<STATS_TYPE> crossValidation(List<ITEM_TYPE> items, int nFolds) throws Exception
items - The items on which to train and test. Good machine learning practice requires that
these items come only from the training data, and not from the test data.nFolds - The number of subsets into which the items should be split. Note that the number of
folds may not be larger than the number of items.Exceptionprotected abstract CollectionReader getCollectionReader(List<ITEM_TYPE> items) throws Exception
CollectionReader from the given items.
This method is called in trainAndTest(List, List) and
crossValidation(List, int) to create readers both for the training data and for the
testing data.items - Items from the training, test or cross-validation sets.CollectionReader that produces CASes for the items.Exceptionprotected List<ITEM_TYPE> selectFoldTestItems(List<ITEM_TYPE> items, int nFolds, int fold)
items - The full list of training items.nFolds - The total number of folds in this cross validation.fold - The index of the fold (0 <= fold < nFolds) whose test items are to be selected.protected List<ITEM_TYPE> selectFoldTrainItems(List<ITEM_TYPE> items, int nFolds, int fold)
items - The full list of training items.nFolds - The total number of folds in this cross validation.fold - The index of the fold (0 <= fold < nFolds) whose training items are to be selected.protected abstract STATS_TYPE test(CollectionReader collectionReader, File directory) throws Exception
collectionReader - The data on which the model should be tested.directory - The directory in which any model files should be written. This method may safely
assume that train(CollectionReader, File) was called on this same directory
before test(CollectionReader, File) was called.Exceptionprotected abstract void train(CollectionReader collectionReader, File directory) throws Exception
collectionReader - The data on which the model should be trained.directory - The directory in which any model files should be written.Exceptionpublic STATS_TYPE trainAndTest(List<ITEM_TYPE> trainItems, List<ITEM_TYPE> testItems) throws Exception
trainItems - The items on which to train.testItems - The items on which to test.ExceptionCopyright © 2014. All rights reserved.