org.duracloud.mill.ltp

Class LoopingTaskProducer<T extends Morsel>

java.lang.Object

org.duracloud.mill.ltp.LoopingTaskProducer<T>

All Implemented Interfaces:

Runnable

public abstract class LoopingTaskProducer<T extends Morsel>
extends Object
implements Runnable

This class is responsible for filling the duplication TaskQueue by looping through all duplication policies for all accounts and spaces and blindly creates duplication tasks. It will create tasks for all items in the the source destination as well as all items in the destination provider but not in the source provider. A notable feature of this task producer is that it attempts to respect a designated maximum task queue size. Once the limit has been reached, the producer will stop. On subsequent runs, the producer will pick up where it left off, starting with the next account,space,set of content items, and duplication store policy. If all content items are visited within a single run before the task queue limit has been reached, the producer will exit. For more information about how this process fits into the whole system of collaborating components, see https://wiki.duraspace.org/display/DSPINT/DuraCloud+Duplication+-+System+Overview

Author:

Daniel Bernstein Date: Nov 5, 2013

Constructor Summary

Constructors

Constructor and Description
LoopingTaskProducer(CredentialsRepo credentialsRepo, StorageProviderFactory storageProviderFactory, org.duracloud.common.queue.TaskQueue taskQueue, StateManager<T> state, int maxTaskQueueSize, Frequency frequency, java.time.LocalTime startTime, NotificationManager notificationManager, LoopingTaskProducerConfigurationManager config)

Method Summary

Modifier and Type	Method and Description
protected void	addToReloadList(T morsel)
protected RunStats	calculateStatTotals(RunStats currentTotals)
protected Queue<T>	createQueue()
protected abstract RunStats	createRunStats()
protected CredentialsRepo	getCredentialsRepo()
protected Frequency	getFrequency()
protected abstract String	getLoopingProducerTypePrefix() A short looping producer type identifier for use with state files.
protected int	getMaxTaskQueueSize()
protected RunStats	getStats(String account)
protected org.duracloud.storage.provider.StorageProvider	getStorageProvider(StorageProviderCredentials creds)
protected org.duracloud.storage.provider.StorageProvider	getStorageProvider(String account, String storeId)
protected org.duracloud.common.queue.TaskQueue	getTaskQueue()
protected abstract void	loadMorselQueueFromSource(Queue<T> morselQueue)
protected abstract void	logCumulativeSessionStats(Map<String,RunStats> runstats, RunStats cumulativeTotals)
protected abstract void	logGlobalncrementalStats(RunStats incrementalTotals)
protected abstract void	logIncrementalStatsByAccount(String account, RunStats stats)
protected abstract void	nibble(Queue<T> queue)
void	run()
protected void	sendEmail(String message, Exception ex)
protected void	sendEmail(String subject, String body)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LoopingTaskProducer

public LoopingTaskProducer(CredentialsRepo credentialsRepo,
                           StorageProviderFactory storageProviderFactory,
                           org.duracloud.common.queue.TaskQueue taskQueue,
                           StateManager<T> state,
                           int maxTaskQueueSize,
                           Frequency frequency,
                           java.time.LocalTime startTime,
                           NotificationManager notificationManager,
                           LoopingTaskProducerConfigurationManager config)

Method Detail

getFrequency

protected Frequency getFrequency()

getCredentialsRepo

protected CredentialsRepo getCredentialsRepo()

getTaskQueue

protected org.duracloud.common.queue.TaskQueue getTaskQueue()

getMaxTaskQueueSize

protected int getMaxTaskQueueSize()

run

public void run()

Specified by:

run in interface Runnable

calculateStatTotals

protected RunStats calculateStatTotals(RunStats currentTotals)

sendEmail

protected void sendEmail(String subject,
                         String body)

createQueue

protected Queue<T> createQueue()

Returns:

addToReloadList

protected void addToReloadList(T morsel)

Parameters:

morsel -

getStats

protected RunStats getStats(String account)

Parameters:

account -

Returns:

getStorageProvider

protected org.duracloud.storage.provider.StorageProvider getStorageProvider(String account,
                                                                            String storeId)

getStorageProvider

protected org.duracloud.storage.provider.StorageProvider getStorageProvider(StorageProviderCredentials creds)

Parameters:

creds -

Returns:

loadMorselQueueFromSource

protected abstract void loadMorselQueueFromSource(Queue<T> morselQueue)

Parameters:

morselQueue -

nibble

protected abstract void nibble(Queue<T> queue)

Parameters:

queue -

createRunStats

protected abstract RunStats createRunStats()

Returns:

logGlobalncrementalStats

protected abstract void logGlobalncrementalStats(RunStats incrementalTotals)

Parameters:

incrementalTotals -

logIncrementalStatsByAccount

protected abstract void logIncrementalStatsByAccount(String account,
                                                     RunStats stats)

Parameters:

account -

stats -

logCumulativeSessionStats

protected abstract void logCumulativeSessionStats(Map<String,RunStats> runstats,
                                                  RunStats cumulativeTotals)

Parameters:

runstats -

cumulativeTotals -

getLoopingProducerTypePrefix

protected abstract String getLoopingProducerTypePrefix()

A short looping producer type identifier for use with state files.

Returns:

sendEmail

protected void sendEmail(String message,
                         Exception ex)

Parameters:

message -

ex -