de.julielab.jcore.ae.jtbd

Class TokenizerApplication

java.lang.Object

de.julielab.jcore.ae.jtbd.TokenizerApplication

public class TokenizerApplication
extends Object

Constructor Summary

Constructors

Constructor and Description
TokenizerApplication()

Method Summary

Modifier and Type	Method and Description
static de.julielab.jcore.ae.jtbd.TokenizerApplication.EvalResult	doEvaluation(ArrayList<String> trainOrgSentences, ArrayList<String> trainTokSentences, ArrayList<String> predictOrgSentences, ArrayList<String> predictTokSentences, ArrayList<String> errors, ArrayList<String> predictions) general evaluation function, is called from doCrossEvaluation or do9010Evaluation.
static void	doPrediction(File inDir, File outDir, String modelFilename) tokenize documents
static void	doTraining(File orgSentencesFile, File tokSentencesFile, String modelFilename) train a model
static void	main(String[] args)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TokenizerApplication

public TokenizerApplication()

Method Detail

doEvaluation

public static de.julielab.jcore.ae.jtbd.TokenizerApplication.EvalResult doEvaluation(ArrayList<String> trainOrgSentences,
                                                                                     ArrayList<String> trainTokSentences,
                                                                                     ArrayList<String> predictOrgSentences,
                                                                                     ArrayList<String> predictTokSentences,
                                                                                     ArrayList<String> errors,
                                                                                     ArrayList<String> predictions)

general evaluation function, is called from doCrossEvaluation or do9010Evaluation.

Parameters:

crf - the crf model

predictOrgSentences -

predictTokSentences -

errors -

predictions -

Returns:

doPrediction

public static void doPrediction(File inDir,
                                File outDir,
                                String modelFilename)
                         throws IOException

tokenize documents

Parameters:

inDir - the directory with the documents to be tokenized

outDir - the directory where the tokenized documents should be written to

modelFile - the model to use for tokenization

Throws:

IOException

doTraining

public static void doTraining(File orgSentencesFile,
                              File tokSentencesFile,
                              String modelFilename)

train a model

Parameters:

orgSentencesFile -

tokSentencesFile -

modelFilename -

main

public static void main(String[] args)
                 throws IOException

Throws:

IOException