Package org.pipecraft.pipes.async.source

Class AsyncStorageMultiFileReaderPipe<T,B>

java.lang.Object

org.pipecraft.pipes.async.AsyncPipe<T>

org.pipecraft.pipes.async.inter.AsyncCompoundPipe<T>

org.pipecraft.pipes.async.source.AsyncStorageMultiFileReaderPipe<T,B>

Type Parameters:

T - The output item data type

B - The remote file metadata object type

All Implemented Interfaces:

Closeable, AutoCloseable, BasePipe

public class AsyncStorageMultiFileReaderPipe<T,B>
extends AsyncCompoundPipe<T>

An async source pipe reading multiple remote binary files from the cloud. Supports file filtering and automatic sharding of the data.

Author:

Eyal Schneider

Nested Class Summary

Nested classes/interfaces inherited from class org.pipecraft.pipes.async.AsyncPipe

AsyncPipe.Status

Constructor Summary

Constructors

Constructor	Description
AsyncStorageMultiFileReaderPipe(StorageMultiFileReaderConfig<T,B> config)	Constructor

Method Summary

Modifier and Type	Method	Description
protected AsyncPipe<T>	createPipeline()

Methods inherited from class org.pipecraft.pipes.async.inter.AsyncCompoundPipe

close, getProgress, start

Methods inherited from class org.pipecraft.pipes.async.AsyncPipe

getStatus, notifyDone, notifyError, notifyNext, setListener

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AsyncStorageMultiFileReaderPipe

public AsyncStorageMultiFileReaderPipe(StorageMultiFileReaderConfig<T,B> config)

Constructor

Parameters:

config - The multi reader config

Method Detail

createPipeline

protected AsyncPipe<T> createPipeline()
                               throws IOException

Specified by:

createPipeline in class AsyncCompoundPipe<T>

Returns:

A new pipeline to represent the logic of this pipe

Throws:

IOException - In case of IO error while building the internal pipeline