Package org.anchoranalysis.experiment.bean.processor

Class ParallelProcessor<T extends InputFromManager,S>

Object

org.anchoranalysis.bean.AnchorBean<JobProcessor<T,S>>

org.anchoranalysis.experiment.bean.processor.JobProcessor<T,S>

org.anchoranalysis.experiment.bean.processor.ParallelProcessor<T,S>

Type Parameters:

T - input-object type

S - shared-state type

All Implemented Interfaces:

ReplaceTask<T,S>

public class ParallelProcessor<T extends InputFromManager,S>
extends JobProcessor<T,S>

Executes jobs in parallel across cores on the system.

Each input is processed in a separate thread on an available core.

Author:

Owen Feehan

Constructor Summary

Constructors

Constructor	Description
ParallelProcessor()

Method Summary

Modifier and Type	Method	Description
protected TaskStatistics	execute(Outputter rootOutputter, List<T> inputs, ParametersExperiment parametersExperiment)	Executes the task on all inputs.
int	getKeepProcessorsFree()	How many processors to avoid using for the tasks.
int	getMaxNumberProcessors()	Theoretical maximum number of (CPU) processors to use in parallel.
int	getNumberGPUProcessors()	How many GPU processors to use when this is possible as a substitute for a CPU processor
int	getShowOngoingJobsLessThan()	When the number of ongoing jobs is less than this threshold, they are shown in event logs.
void	setKeepProcessorsFree(int keepProcessorsFree)	How many processors to avoid using for the tasks.
void	setMaxNumberProcessors(int maxNumberProcessors)	Theoretical maximum number of (CPU) processors to use in parallel.
void	setNumberGPUProcessors(int numberGPUProcessors)	How many GPU processors to use when this is possible as a substitute for a CPU processor
void	setShowOngoingJobsLessThan(int showOngoingJobsLessThan)	When the number of ongoing jobs is less than this threshold, they are shown in event logs.

Methods inherited from class org.anchoranalysis.experiment.bean.processor.JobProcessor

executeLogStatistics, getTask, hasVeryQuickPerInputExecution, isInputCompatibleWith, isSuppressExceptions, replaceTask, setSuppressExceptions, setTask

Methods inherited from class org.anchoranalysis.bean.AnchorBean

checkMisconfigured, describeBean, describeChildren, duplicateBean, fields, findFieldsOfClass, getBeanName, getLocalPath, localise, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Details

ParallelProcessor

public ParallelProcessor()

Method Details

execute

protected TaskStatistics execute(Outputter rootOutputter, List<T> inputs, ParametersExperiment parametersExperiment) throws ExperimentExecutionException

Description copied from class: JobProcessor

Executes the task on all inputs.

It is expected that elements are removed from inputs as they are consumed so as to allow garbage-collection of these items before all jobs are processed (as the list might be quite large).

Specified by:

execute in class JobProcessor<T extends InputFromManager,S>

Parameters:

rootOutputter - an outputter, bound to the base (root) output directory into which outputted files are written.

inputs - the inputs to apply task on.

parametersExperiment - parameters that exist pertaining to the experiment that is underway.

Returns:

statistics about the success/failure/execution-time etc. of applying the task to inputs.

Throws:

ExperimentExecutionException - if anything goes wrong executing the experiment (but not necessarily if a processing a particular input fails when suppressExceptions == true.

getMaxNumberProcessors

public int getMaxNumberProcessors()

Theoretical maximum number of (CPU) processors to use in parallel.

In practice, the effective maximum is min(maxNumberProcessors,numberAvailableProcessors), usually lower.

setMaxNumberProcessors

public void setMaxNumberProcessors(int maxNumberProcessors)

Theoretical maximum number of (CPU) processors to use in parallel.

In practice, the effective maximum is min(maxNumberProcessors,numberAvailableProcessors), usually lower.

getShowOngoingJobsLessThan

public int getShowOngoingJobsLessThan()

When the number of ongoing jobs is less than this threshold, they are shown in event logs. 0 disables.

setShowOngoingJobsLessThan

public void setShowOngoingJobsLessThan(int showOngoingJobsLessThan)

When the number of ongoing jobs is less than this threshold, they are shown in event logs. 0 disables.

getKeepProcessorsFree

public int getKeepProcessorsFree()

How many processors to avoid using for the tasks.

When using the maximum available number of processors, a certain amount are deliberately not used, to save them for other tasks on the operating system. This is particularly valuable on a desktop PC where other tasks (e.g web browsing) may be ongoing during processing.

setKeepProcessorsFree

public void setKeepProcessorsFree(int keepProcessorsFree)

How many processors to avoid using for the tasks.

When using the maximum available number of processors, a certain amount are deliberately not used, to save them for other tasks on the operating system. This is particularly valuable on a desktop PC where other tasks (e.g web browsing) may be ongoing during processing.

getNumberGPUProcessors

public int getNumberGPUProcessors()

How many GPU processors to use when this is possible as a substitute for a CPU processor

setNumberGPUProcessors

public void setNumberGPUProcessors(int numberGPUProcessors)

How many GPU processors to use when this is possible as a substitute for a CPU processor