Package org.corpus_tools.pepper.modules.coreModules

Class TextImporter

java.lang.Object

org.corpus_tools.pepper.impl.PepperModuleImpl

org.corpus_tools.pepper.impl.PepperImporterImpl

org.corpus_tools.pepper.modules.coreModules.TextImporter

All Implemented Interfaces:

Thread.UncaughtExceptionHandler, PepperImporter, PepperModule

public class TextImporter
extends PepperImporterImpl
implements PepperImporter

Maps a file-structure to a corpus-structure and reads the contained txt files to map their content to STextualDS objects.

Author:

Florian Zipser

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	TextImporter.TextMapper	Reads the content of txt files and creates a STextualDS object for each.

Field Summary

Fields

Modifier and Type	Field	Description
static String	FORMAT_NAME
static String	FORMAT_VERSION
static String	MODULE_NAME

Fields inherited from class org.corpus_tools.pepper.impl.PepperImporterImpl

corpusDesc

Fields inherited from class org.corpus_tools.pepper.impl.PepperModuleImpl

isMultithreaded, logger, moduleController, resources, saltProject, sCorpusGraph, symbolicName, temproraries

Fields inherited from interface org.corpus_tools.pepper.modules.PepperImporter

NEGATIVE_FILE_EXTENSION_MARKER

Fields inherited from interface org.corpus_tools.pepper.modules.PepperModule

ENDING_ALL_FILES, ENDING_FOLDER, ENDING_LEAF_FOLDER, ENDING_TAB, ENDING_TXT, ENDING_XML

Constructor Summary

Constructors

Constructor	Description
TextImporter()	Specifies the separator, which has to be set between to the texts of two token.

Method Summary

Modifier and Type	Method	Description
PepperMapper	createPepperMapper(org.corpus_tools.salt.graph.Identifier sElementId)	Creates a mapper OVERRIDE THIS METHOD FOR CUSTOMIZED MAPPING.
SelfTestDesc	getSelfTestDesc()	This method is called by the Pepper framework to run an integration test for module.
Double	isImportable(org.eclipse.emf.common.util.URI corpusPath)	{@inheritDoc PepperImporter#isImportable(URI)}

Methods inherited from class org.corpus_tools.pepper.impl.PepperImporterImpl

addSupportedFormat, getCorpusDesc, getCorpusEndings, getDocumentEndings, getIdentifier2ResourceTable, getIgnoreEndings, getSupportedFormats, importCorpusStructure, importCorpusStructureRec, readXMLResource, sampleFileContent, setCorpusDesc, setCorpusPathResolver, setTypeOfResource, start

Methods inherited from class org.corpus_tools.pepper.impl.PepperModuleImpl

activate, done, done, end, getComponentContext, getCorpusGraph, getDesc, getDocumentId2DC, getFingerprint, getMapperControllers, getMapperThreadGroup, getModuleController, getModuleType, getName, getProgress, getProgress, getProperties, getResources, getSaltProject, getStartProblems, getSupplierContact, getSupplierHomepage, getSymbolicName, getTemproraries, getVersion, isMultithreaded, isReadyToStart, proposeImportOrder, setCorpusGraph, setDesc, setIsMultithreaded, setMapperThreadGroup, setName, setPepperModuleController, setPepperModuleController_basic, setProperties, setResources, setSaltProject, setSupplierContact, setSupplierHomepage, setSymbolicName, setTemproraries, setVersion, start, toString, uncaughtException

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.corpus_tools.pepper.modules.PepperImporter

addSupportedFormat, getCorpusDesc, getCorpusEndings, getDocumentEndings, getIdentifier2ResourceTable, getIgnoreEndings, getSupportedFormats, importCorpusStructure, setCorpusDesc, setTypeOfResource

Methods inherited from interface org.corpus_tools.pepper.modules.PepperModule

done, done, end, getComponentContext, getCorpusGraph, getDesc, getFingerprint, getModuleController, getModuleType, getName, getProgress, getProgress, getProperties, getResources, getSaltProject, getStartProblems, getSupplierContact, getSupplierHomepage, getSymbolicName, getTemproraries, getVersion, isMultithreaded, isReadyToStart, proposeImportOrder, setCorpusGraph, setDesc, setIsMultithreaded, setPepperModuleController, setPepperModuleController_basic, setProperties, setResources, setSaltProject, setSupplierContact, setSupplierHomepage, setSymbolicName, setTemproraries, setVersion, start, start

Field Detail

MODULE_NAME

public static final String MODULE_NAME

See Also:

Constant Field Values

FORMAT_NAME

public static final String FORMAT_NAME

See Also:

Constant Field Values

FORMAT_VERSION

public static final String FORMAT_VERSION

See Also:

Constant Field Values

Constructor Detail

TextImporter

public TextImporter()

Specifies the separator, which has to be set between to the texts of two token.

Method Detail

isImportable

public Double isImportable(org.eclipse.emf.common.util.URI corpusPath)

Description copied from class: PepperImporterImpl

{@inheritDoc PepperImporter#isImportable(URI)}

Specified by:

isImportable in interface PepperImporter

Overrides:

isImportable in class PepperImporterImpl

Returns:

1 if corpus is importable, 0 if corpus is not importable, 0 < X < 1, if no definitiv answer is possible, null if method is not overridden

getSelfTestDesc

public SelfTestDesc getSelfTestDesc()

Description copied from interface: PepperModule

This method is called by the Pepper framework to run an integration test for module. When the method returns null, it means that no integration test is supported. Otherwise, the SelfTestDesc object needs to provide an input corpus path and an output corpus path. When this module is:

• an importer: SelfTestDesc.getInputCorpusPath() should contain the format to be imported. SelfTestDesc.getExpectedCorpusPath() should contain the expected salt project (for control).
• a manipulator: SelfTestDesc.getInputCorpusPath() should contain a salt project which is the module's input. SelfTestDesc.getExpectedCorpusPath() should contain the expected salt project (for control).
• an exporter: SelfTestDesc.getInputCorpusPath() should contain a salt project which is the module's input. SelfTestDesc.getExpectedCorpusPath() should contain the expected corpus in output format.

The simplest way to create a test description is:

 return new IntegrationTestDesc(inputPath, outputPath);
 

When this module is an importer or a manipulator the method SelfTestDesc.compare(SaltProject, SaltProject) is called to compare output salt project with expected salt project. When the module is an exporter the method SelfTestDesc.compare(URI, URI) is called to compare the created output folder with an expected one. By default this method checks whether the file structure and each file is equal.

Specified by:

getSelfTestDesc in interface PepperModule

Overrides:

getSelfTestDesc in class PepperModuleImpl

Returns:

test description

createPepperMapper

public PepperMapper createPepperMapper(org.corpus_tools.salt.graph.Identifier sElementId)

Creates a mapper OVERRIDE THIS METHOD FOR CUSTOMIZED MAPPING. This method creates a customized PepperMapper object and returns it. You can here do some additional initialisations. Thinks like setting the Identifier of the SDocument or SCorpus object and the URI resource is done by the framework (or more in detail in method PepperModule.start()). The parameter sElementId, if a PepperMapper object should be created in case of the object to map is either an SDocument object or an SCorpus object of the mapper should be initialized differently.
Note: Override this method.

Specified by:

createPepperMapper in interface PepperModule

Overrides:

createPepperMapper in class PepperModuleImpl

Parameters:

sElementId - Identifier of the SCorpus or SDocument to be processed.

Returns:

PepperMapper object to do the mapping task for object connected to given Identifier