org.icij.extract.extractor

Class Extractor

java.lang.Object

org.icij.extract.extractor.Extractor

public class Extractor
extends Object

A reusable class that sets up Tika parsers based on runtime options.

Since:

1.0.0-beta

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Extractor.EmbedHandling
static class	Extractor.OutputFormat

Constructor Summary

Constructors

Constructor and Description
Extractor() Create a new extractor, which will OCR images by default if Tesseract is available locally, extract inline images from PDF files and OCR them and use PDFBox's non-sequential PDF parser.

Method Summary

Modifier and Type	Method and Description
Extractor	configure(Options<String> options)
void	disableOcr() Disable OCR.
Reader	extract(TikaDocument tikaDocument) This method will wrap the given TikaDocument in a TikaInputStream and return a Reader which can be used to initiate extraction on demand.
void	extract(TikaDocument tikaDocument, Spewer spewer) Extract and spew content from a document.
void	extract(TikaDocument tikaDocument, Spewer spewer, Reporter reporter) Extract and spew content from a document.
protected Reader	extract(TikaDocument tikaDocument, org.apache.tika.io.TikaInputStream input) Create a pull-parser from the given TikaInputStream.
Extractor.EmbedHandling	getEmbedHandling() Get the embed handling mode.
Path	getEmbedOutputPath() Get the output directory path for embed files.
Extractor.OutputFormat	getOutputFormat() Get the extraction output format.
void	setDigestAlgorithms(org.apache.tika.parser.utils.CommonsDigester.DigestAlgorithm... digestAlgorithms)
void	setEmbedHandling(Extractor.EmbedHandling embedHandling) Set the embed handling mode.
void	setEmbedOutputPath(Path embedOutput) Set the output directory path for embed files.
void	setOcrLanguage(String ocrLanguage) Set the languages used by Tesseract.
void	setOcrTimeout(java.time.Duration duration) Instructs Tesseract to attempt OCR for no longer than the given duration.
void	setOutputFormat(Extractor.OutputFormat outputFormat) Set the output format.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Extractor

public Extractor()

Create a new extractor, which will OCR images by default if Tesseract is available locally, extract inline images from PDF files and OCR them and use PDFBox's non-sequential PDF parser.

Method Detail

configure

public Extractor configure(Options<String> options)

setOutputFormat

public void setOutputFormat(Extractor.OutputFormat outputFormat)

Set the output format.

Parameters:

outputFormat - the output format

getOutputFormat

public Extractor.OutputFormat getOutputFormat()

Get the extraction output format.

Returns:

the output format

setEmbedHandling

public void setEmbedHandling(Extractor.EmbedHandling embedHandling)

Set the embed handling mode.

Parameters:

embedHandling - the embed handling mode

getEmbedHandling

public Extractor.EmbedHandling getEmbedHandling()

Get the embed handling mode.

Returns:

the embed handling mode.

setEmbedOutputPath

public void setEmbedOutputPath(Path embedOutput)

Set the output directory path for embed files.

Parameters:

embedOutput - the embed output path

getEmbedOutputPath

public Path getEmbedOutputPath()

Get the output directory path for embed files.

Returns:

the embed output path.

setOcrLanguage

public void setOcrLanguage(String ocrLanguage)

Set the languages used by Tesseract.

Parameters:

ocrLanguage - the languages to use, for example "eng" or "ita+spa"

setOcrTimeout

public void setOcrTimeout(java.time.Duration duration)

Instructs Tesseract to attempt OCR for no longer than the given duration.

Parameters:

duration - the duration before timeout

setDigestAlgorithms

public void setDigestAlgorithms(org.apache.tika.parser.utils.CommonsDigester.DigestAlgorithm... digestAlgorithms)

disableOcr

public void disableOcr()

Disable OCR. This method only has an effect if Tesseract is installed.

extract

public Reader extract(TikaDocument tikaDocument)
               throws IOException

This method will wrap the given TikaDocument in a TikaInputStream and return a Reader which can be used to initiate extraction on demand. Internally, this method uses TikaInputStream.get(java.io.InputStream, org.apache.tika.io.TemporaryResources) which ensures that the resource name and content length metadata properties are set automatically.

Parameters:

tikaDocument - the file to extract from

Returns:

A Reader that can be used to read extracted text on demand.

Throws:

IOException

extract

public void extract(TikaDocument tikaDocument,
                    Spewer spewer)
             throws IOException

Extract and spew content from a document. Internally, as with extract(TikaDocument), this method creates a TikaInputStream from the path of the given document.

Parameters:

tikaDocument - document to extract from

spewer - endpoint to write to

Throws:

IOException - if there was an error reading or writing the document

extract

public void extract(TikaDocument tikaDocument,
                    Spewer spewer,
                    Reporter reporter)

Extract and spew content from a document. This method is the same as extract(TikaDocument, Spewer) with the exception that the document will be skipped if the reporter returns false for a call to Reporter.skip(TikaDocument). If the document is not skipped, then the result of the extraction is passed to the reporter in a call to Reporter.save(TikaDocument, ExtractionStatus, Exception).

Parameters:

tikaDocument - document to extract from

spewer - endpoint to write to

reporter - used to check whether the document should be skipped and save extraction status

extract

protected Reader extract(TikaDocument tikaDocument,
                         org.apache.tika.io.TikaInputStream input)
                  throws IOException

Create a pull-parser from the given TikaInputStream.

Parameters:

input - the stream to extract from

tikaDocument - file that is being extracted from

Returns:

A pull-parsing reader.

Throws:

IOException