Uses of Interface
org.pipecraft.pipes.BasePipe

Packages that use BasePipe

Package	Description
org.pipecraft.pipes.async
org.pipecraft.pipes.async.inter
org.pipecraft.pipes.async.inter.distributed
org.pipecraft.pipes.async.source
org.pipecraft.pipes.sync
org.pipecraft.pipes.sync.inter
org.pipecraft.pipes.sync.inter.join
org.pipecraft.pipes.sync.inter.reduct
org.pipecraft.pipes.sync.inter.sample
org.pipecraft.pipes.sync.source
org.pipecraft.pipes.terminal
org.pipecraft.pipes.utils

Uses of BasePipe in org.pipecraft.pipes.async

Classes in org.pipecraft.pipes.async that implement BasePipe

Modifier and Type	Class	Description
class	AsyncPipe<T>	A pipe that produces items asynchronously.

Uses of BasePipe in org.pipecraft.pipes.async.inter

Classes in org.pipecraft.pipes.async.inter that implement BasePipe

Modifier and Type	Class	Description
class	AsyncCallbackPipe<T>	A no-op async pipe that "spies" on data passing through it.
class	AsyncCompoundPipe<T>	A pipe encapsulating a pipeline.
class	AsyncFilterPipe<T>	An async version of the filter pipe (See FilterPipe.
class	AsyncFlexibleMapPipe<S,T>	An async version of the flexible mapper pipe (See FlexibleMapPipe.
class	AsyncHeadPipe<T>	An async version of the head pipe (See HeadPipe.
class	AsyncMapPipe<S,T>	An async version of the mapper pipe (See MapPipe.
class	AsyncProgressPipe<T>	A no-op async pipe that reports progress.
class	AsyncTimeoutPipe<T>	An async pipe enforcing a timeout on completion of its async input pipe.
class	AsyncUnionPipe<T>	An intermediate async pipe fed by a set of other async pipes.
class	SyncToAsyncPipe<T>	A pipe converting multiple sync pipes into a single async one.

Uses of BasePipe in org.pipecraft.pipes.async.inter.distributed

Classes in org.pipecraft.pipes.async.inter.distributed that implement BasePipe

Modifier and Type	Class	Description
class	DistributedShufflerPipe<T>	An async pipe that takes input and shuffle it across multiple workers.

Uses of BasePipe in org.pipecraft.pipes.async.source

Classes in org.pipecraft.pipes.async.source that implement BasePipe

Modifier and Type	Class	Description
class	AsyncEmptyPipe<T>	An async source pipe returning no data.
class	AsyncMultiFileReaderPipe<T>	An async source pipe reading multiple local binary files.
class	AsyncSeqGenPipe<T>	An async source pipe, producing a fixed number of items using a fixed number of threads.
class	AsyncStorageMultiFileReaderPipe<T,B>	An async source pipe reading multiple remote binary files from the cloud.

Uses of BasePipe in org.pipecraft.pipes.sync

Subinterfaces of BasePipe in org.pipecraft.pipes.sync

Modifier and Type	Interface	Description
interface	Pipe<T>	The interface to be implemented by synchronous pipes, which are passive and items are "pulled" from them.

Uses of BasePipe in org.pipecraft.pipes.sync.inter

Classes in org.pipecraft.pipes.sync.inter that implement BasePipe

Modifier and Type	Class	Description
class	AsyncToSyncPipe<T>	A pipe which acts as a converter from async pipe/s to a sync pipe.
class	CallbackPipe<T>	A pipe invoking a callback method per item passed through it.
class	CompoundPipe<T>	A pipe encapsulating a pipeline.
class	ConcatPipe<T>	An intermediate pipe consisting of concatenating the contents of a series of pipes
class	CountPipe	A pipe counting the items in the input pipe, and returning the count as a single output item.
class	CSVMapperPipe<T>	An intermediate pipe that parses strings in CSV format, and binds each line to an object
class	DedupPipe<T>	Uses item equality (equals() method) in input pipe items for performing a dedup operation.
class	DelegatePipe<T>	A pipe delegating all work to another pipe given in the constructor.
class	FilterBasePipe<T>	A base-class pipe implementation for pipes that filter an input pipe using some criteria.
class	FilterPipe<T>	A pipe filtering an input pipe by some predicate on the pipe items.
class	FlexibleMapPipe<S,T>	An intermediate pipe that maps an input item into zero or more items of another type.
class	GrouperPipe<T>	Splits the input items into different families, and emits items from the same family sequentially.
class	HeadPipe<T>	A pipe returning the first K items of its input pipe.
class	IntermediateBinFileWriterPipe<T>	The intermediate version of BinFileWriterPipe.
class	IntermediateSharderBySeqPipe<T>	An intermediate version of SharderBySeqPipe
class	IntermediateTxtFileWriterPipe<T>	A pipe write to a text file any item passed through it.
class	MapPipe<S,T>	A pipe transforming each item in the input pipe into another, possibly with a different type.
class	MultiSortedBasePipe<T>	A base class for pipes performing some set operation on a collection of sorted input pipes.
class	OrderValidationPipe<T>	An intermediate pipe that validates the order of the consumed elements by a comparator
class	ProgressPipe<T>	A no-op pipe with only one side effect - progress tracking.
class	SkipPipe<T>	A pipe skipping the first K items of its input pipe.
class	SortedIntersectionPipe<T>	An intermediate pipe performing set intersection operation on a collection of sorted input pipes.
class	SortedMergePipe<T>	An intermediate pipe sort-merging the contents of a collection of sorted input pipes.
class	SortedSubtractionPipe<T>	An intermediate pipe performing a set subtraction operation on two given pipes (i.e.
class	SortedUnionPipe<T>	An intermediate pipe performing set union operation on a collection of sorted input pipes.
class	SortPipe<T>	An intermediate pipe performing a sort on an input pipe, without deduping.
class	TopKPipe<T>	A pipe returning the top K items of the input pipe, according to some predefined order relation.

Uses of BasePipe in org.pipecraft.pipes.sync.inter.join

Classes in org.pipecraft.pipes.sync.inter.join that implement BasePipe

Modifier and Type	Class	Description
class	HashJoinPipe<K,L,R>	A pipe performing a join operation between a 'left' pipe of type L, and a list of 'right' pipes of type R.
class	LookupJoinPipe<K,L,R>	A pipe performing a join operation between a 'left' pipe of type L, and a list of 'right' pipes of type R.
class	SortedJoinPipe<K,L,R>	A pipe performing a join operation between a 'left' pipe of type L, and a list of 'right' pipes of type R.

Uses of BasePipe in org.pipecraft.pipes.sync.inter.reduct

Classes in org.pipecraft.pipes.sync.inter.reduct that implement BasePipe

Modifier and Type	Class	Description
class	HashReductorPipe<I,O>	Scans the input pipe and performs a reduction operation on families of items based on some discriminating property.
class	ListReductorPipe<S,T>	Scans the input pipe and groups sequential items based on some discriminating property.
class	SequenceReductorPipe<I,O>	Scans the input pipe and performs a reduction operation on a sequence of items with some discriminating property.

Uses of BasePipe in org.pipecraft.pipes.sync.inter.sample

Classes in org.pipecraft.pipes.sync.inter.sample that implement BasePipe

Modifier and Type	Class	Description
class	ExactSamplerPipe<T>	A pipe sampling exactly m items of an input pipe containing n items, where n is known in advance.
class	PortionSamplerPipe<T>	A pipe sampling about x% of the incoming items.

Uses of BasePipe in org.pipecraft.pipes.sync.source

Classes in org.pipecraft.pipes.sync.source that implement BasePipe

Modifier and Type	Class	Description
class	BinInputReaderPipe<T>	A source pipe reading items from a given binary input (file or input stream).
class	CollectionReaderPipe<T>	A pipe based on a given collection of items.
class	EmptyPipe<T>	A source pipe with no data in it.
class	ErrorPipe<T>	A source pipe with no data in it, which only produces a predefined PipeException.
class	InputStreamPipe<T>	A base class for source pipes based on some input stream.
class	IteratorReaderPipe<T>	A pipe based on an iterator.
class	JdbcQueryResultsPipe<T>	A source pipe reading rows from a DB query result set, and converting them to some entity type
class	MultiFileReaderPipe<T>	A source pipe reading multiple local binary files.
class	MultiTxtFileReaderPipe	Reads data from multiple files in local disk under some folder, as if they were concatenated using some predefined order.
class	QueueReaderPipe<T>	A source pipe reading the contents of a BlockingQueue.
class	ReaderPipe	A source pipe that reads lines from a given reader.
class	SeqGenPipe<T>	A pipe based on a given sequence generator function.
class	StorageMultiFileReaderPipe<T,B>	A source pipe reading multiple remote binary files from the cloud.
class	StorageMultiTxtFileReaderPipe<B>	Reads data from multiple files in cloud storage under some folder, as if they were concatenated using some predefined order.
class	StorageTxtFileFetcherReaderPipe	A source pipe providing the lines of a text file from cloud storage.
class	StorageTxtFileReaderPipe	A source pipe providing the lines of a text file in some Storage, in a streaming manner.
class	TxtFileReaderPipe	A source pipe reading lines from a given local text file or input stream containing text content.
class	URLTxtReaderPipe	A source pipe providing the lines of a text resource defined by a URL.

Uses of BasePipe in org.pipecraft.pipes.terminal

Classes in org.pipecraft.pipes.terminal that implement BasePipe

Modifier and Type	Class	Description
class	AsyncCollectionWriterPipe<T>	A terminal async pipe writing all contents from an input pipe into a given collection.
class	AsyncConsumerPipe<T>	Consumes the contents of an AsyncPipe, blocking in the start() method until all data is read.
class	AsyncEnqueuingSharderPipe<T>	A terminal pipe that receives an async pipe as input, and shards the contents of the input pipe into multiple queues according to some sharding criteria based on item values.
class	AsyncSharderByHashPipe<T>	An async sharder pipe, writing items to files based on the hash value of individual items.
class	AsyncSharderPipe<T>	A terminal pipe that receives an async pipe as input, and splits the contents of the input pipe into multiple files according to some sharding criteria based on individual items.
class	BinFileWriterPipe<T>	A terminal pipe writing items to a local file in a binary format.
class	CollectionWriterPipe<T>	A terminal pipe writing all contents from an input pipe into a given collection.
class	CompoundTerminalPipe	A terminal pipe encapsulating a pipeline.
class	ConsumerPipe<T>	A terminal pipe that consumes all contents of the input pipe in a synchronous manner.
class	ParallelConsumerPipe	A terminal pipe that consumes multiple input pipes in a parallel manner, with a configurable number of threads.
class	PercentilePipe<T,C extends Comparable<C>>	A terminal pipe that finds a given percentile p in the input pipe.
class	QueueWriterPipe<T>	A terminal pipe writing items to a BlockingQueue.
class	SharderByHashPipe<T>	A terminal pipe that splits the contents of the input pipe into multiple files, according to a hash on a some feature of the item.
class	SharderByItemPipe<T>	A terminal pipe that splits the contents of the input pipe into multiple files, according to some sharding criteria based on each item.
class	SharderBySeqPipe<T>	A terminal pipe that splits the contents of the input pipe according to some criteria which breaks the input pipe into disjoint contiguous sequences.
class	StorageTxtFileWriterPipe	A terminal pipe writing text lines from the input pipe into a remote cloud storage file.
class	TerminalPipe	Base class for terminal pipes, i.e.
class	TxtFileWriterPipe	A terminal pipe writing text lines from the input pipe into a file.
class	WriterPipe	A terminal pipe writing textual items to a given Writer, one item per line.

Constructors in org.pipecraft.pipes.terminal with parameters of type BasePipe

Constructor	Description
ParallelConsumerPipe(int parallelism, BasePipe... inputs)	Constructor

Constructor parameters in org.pipecraft.pipes.terminal with type arguments of type BasePipe

Constructor	Description
ParallelConsumerPipe(int parallelism, Collection<? extends BasePipe> inputs)	Constructor
ParallelConsumerPipe(int parallelism, Collection<? extends BasePipe> inputs, Runnable terminationAction)	Constructor

Uses of BasePipe in org.pipecraft.pipes.utils

Methods in org.pipecraft.pipes.utils with parameters of type BasePipe

Modifier and Type	Method	Description
static void	PipeUtils.close(BasePipe... pipes)	Closes the given pipes, ignoring errors.

Method parameters in org.pipecraft.pipes.utils with type arguments of type BasePipe

Modifier and Type	Method	Description
static void	PipeUtils.close(Collection<? extends BasePipe> pipes)	Closes the given pipes, ignoring errors.
static float	PipeUtils.getAverageProgress(Collection<? extends BasePipe> pipes)
static float	PipeUtils.getMaxProgress(Collection<? extends BasePipe> pipes)
static float	PipeUtils.getMinProgress(Collection<? extends BasePipe> pipes)