Package org.pipecraft.infra.concurrent

Class ParallelTaskProcessor

java.lang.Object

org.pipecraft.infra.concurrent.ParallelTaskProcessor

public class ParallelTaskProcessor
extends Object

Processes a heavy task (typically with IO involved) in parallel. The input is a set of task items that have no inter-dependencies and therefore can be processed in parallel. The caller supplies both the item processor and the items to work on.

Author:

Eyal Schneider

Constructor Summary

Constructors

Constructor	Description
ParallelTaskProcessor()

Method Summary

Modifier and Type	Method	Description
static <T> void	run(Collection<T> items, int parallelism, Consumer<? super T> processor)	Processed the items in parallel by running a given processor on them.
static <T> void	run(ExecutorService ex, Collection<T> items, Consumer<? super T> processor)	Processed the items in parallel, by running a given processor on them, using the provided executor.
static <T,E extends Exception> void	runFailable(Collection<T> items, int parallelism, FailableInterruptibleConsumer<? super T,E> processor)	Processed the items in parallel by running a given processor on them.
static <T,E extends Exception> void	runFailable(ExecutorService ex, Collection<T> items, FailableInterruptibleConsumer<? super T,E> processor)	Processed the items in parallel, by running a given processor on them, using the provided executor.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ParallelTaskProcessor

public ParallelTaskProcessor()

Method Detail

runFailable

public static <T,E extends Exception> void runFailable(ExecutorService ex,
                                                             Collection<T> items,
                                                             FailableInterruptibleConsumer<? super T,E> processor)
                                                      throws E extends Exception,
                                                             InterruptedException

Processed the items in parallel, by running a given processor on them, using the provided executor. The processor here is allowed to throw checked exceptions, of type E. Notes: 1) In case of an error while processing one of the items, the method tries to exit as soon as possible. 2) The method returns only after making sure there aren't any pending items. 3) It follows from (1) and (2) that the processor provided here should preferably be interruptible. This way we can exit faster when an error occurs in any of the items.

Parameters:

ex - The executor to use for running the processor on. It is recommended that the number of background threads will be at least the value of the parallelism parameter.

items - The items to be processed. Should be independent of each other, and must be thread safe for concurrent handling by the supplied processor.

processor - The task processor. Must be thread safe. Ideally reacts to interruption requests.

Throws:

E - In case that the processor failed on some item with some error.

InterruptedException - In case that the current thread is interrupted. Note that by the time the exception is thrown, all tasks are guaranteed to be canceled/complete.

E extends Exception

runFailable

public static <T,E extends Exception> void runFailable(Collection<T> items,
                                                             int parallelism,
                                                             FailableInterruptibleConsumer<? super T,E> processor)
                                                      throws E extends Exception,
                                                             InterruptedException

Processed the items in parallel by running a given processor on them. Spawns threads according to the supplied parallelism level. The processor here is allowed to throw checked exceptions, of type E. Notes: 1) In case of an error while processing one of the items, the method tries to exit as soon as possible. 2) The method returns only after making sure there aren't any pending items. 3) It follows from (1) and (2) that the processor provided here should preferably be interruptible. This way we can exit faster when an error occurs in any of the items.

Parameters:

items - The items to be processed. Should be independent of each other, and must be thread safe for concurrent handling by the supplied processor.

parallelism - The number of threads to process the items

processor - The task processor. Must be thread safe. Ideally reacts to interruption requests.

Throws:

RuntimeException - In case of an unchecked error while running the processor on some item

InterruptedException - In case that the current thread is interrupted. Note that by the time the exception is thrown, all tasks are guaranteed to be canceled/complete.

E extends Exception

run

public static <T> void run(ExecutorService ex,
                           Collection<T> items,
                           Consumer<? super T> processor)
                    throws InterruptedException

Processed the items in parallel, by running a given processor on them, using the provided executor. The processor here is not allowed to throw checked exceptions. Notes: 1) In case of an error while processing one of the items, the method tries to exit as soon as possible. 2) The method returns only after making sure there aren't any pending items. 3) It follows from (1) and (2) that the processor provided here should preferably be interruptible. This way we can exit faster when an error occurs in any of the items.

Parameters:

ex - The executor to use for running the processor on. It is recommended that the number of background threads will be at least the value of the parallelism parameter.

items - The items to be processed. Should be independent of each other, and must be thread safe for concurrent handling by the supplied processor.

processor - The task processor. Must be thread safe. Ideally reacts to interruption requests.

Throws:

RuntimeException - In case of an unchecked error while running the processor on some item

InterruptedException - In case that the current thread is interrupted. Note that by the time the exception is thrown, all tasks are guaranteed to be canceled/complete.

run

public static <T> void run(Collection<T> items,
                           int parallelism,
                           Consumer<? super T> processor)
                    throws RuntimeException,
                           InterruptedException

Processed the items in parallel by running a given processor on them. Spawns threads according to the supplied parallelism level. The processor here is not allowed to throw checked exceptions. Notes: 1) In case of an error while processing one of the items, the method tries to exit as soon as possible. 2) The method returns only after making sure there aren't any pending items. 3) It follows from (1) and (2) that the processor provided here should preferably be interruptible. This way we can exit faster when an error occurs in any of the items.

Parameters:

items - The items to be processed. Should be independent of each other, and must be thread safe for concurrent handling by the supplied processor.

parallelism - The number of threads to process the items

processor - The task processor. Must be thread safe. Ideally reacts to interruption requests.

Throws:

RuntimeException - In case of an unchecked error while running the processor on some item

InterruptedException - In case that the current thread is interrupted. Note that by the time the exception is thrown, all tasks are guaranteed to be canceled/complete.