Package org.lockss.laaws.crawler.impl.pluggable

Class CmdLineCrawl

java.lang.Object

org.lockss.laaws.crawler.impl.pluggable.PluggableCrawl

org.lockss.laaws.crawler.impl.pluggable.CmdLineCrawl

public class CmdLineCrawl
extends PluggableCrawl

A class to wrap a single CommandLineCrawl

Nested Class Summary

Nested classes/interfaces inherited from class org.lockss.laaws.crawler.impl.pluggable.PluggableCrawl

PluggableCrawl.PluggableCrawlerStatus

Field Summary

Fields

Modifier and Type	Field	Description
protected static Pattern	bytesPattern
protected List<String>	command
protected CmdLineCrawler	crawler
protected String	errorLogLevel
protected static Pattern	errorPattern
protected String	outputLogLevel
protected static Pattern	successPattern
protected String	threadName
protected File	tmpDir	The temp directory used to store any files.
protected static Pattern	urlPattern

Fields inherited from class org.lockss.laaws.crawler.impl.pluggable.PluggableCrawl

au, crawlDesc, crawlerConfig, crawlerStatus, crawlJob

Constructor Summary

Constructors

Constructor	Description
CmdLineCrawl(CmdLineCrawler crawler, org.lockss.plugin.ArchivalUnit au, org.lockss.util.rest.crawler.CrawlJob crawlJob)	Instantiates a new Cmd line crawl.

Method Summary

Modifier and Type	Method	Description
static long	extractBytes(String str)
static List<String>	extractUrls(String text)	Returns a list with all links contained in the input
List<String>	getCommand()	Gets command.
protected List<String>	getReqUrls()
org.lockss.daemon.LockssRunnable	getRunnable()
protected List<String>	getStems()
File	getTmpDir()	Gets tmp dir.
Collection<File>	getWarcFiles(List<String> exts)
void	parseLine(String pre, String line)
org.lockss.crawler.CrawlerStatus	startCrawl()	Enqueue a crawl request.
org.lockss.crawler.CrawlerStatus	stopCrawl()	Stop crawl crawler status.
String	toString()

Methods inherited from class org.lockss.laaws.crawler.impl.pluggable.PluggableCrawl

generateKey, getAu, getAuId, getCrawlDesc, getCrawlerConfig, getCrawlerId, getCrawlerStatus, getCrawlKey, getCrawlKind, getCrawlStatus, getJobStatus, setCrawlerStatus

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

crawler

protected CmdLineCrawler crawler

threadName

protected String threadName

command

protected List<String> command

tmpDir

protected File tmpDir

The temp directory used to store any files.

outputLogLevel

protected String outputLogLevel

errorLogLevel

protected String errorLogLevel

successPattern

protected static Pattern successPattern

errorPattern

protected static Pattern errorPattern

urlPattern

protected static Pattern urlPattern

bytesPattern

protected static Pattern bytesPattern

Constructor Details

CmdLineCrawl

public CmdLineCrawl(CmdLineCrawler crawler,
 org.lockss.plugin.ArchivalUnit au,
 org.lockss.util.rest.crawler.CrawlJob crawlJob)

Instantiates a new Cmd line crawl.

Parameters:

crawler - the crawler for this crawl

crawlJob - the job for this crawl

Method Details

startCrawl

public org.lockss.crawler.CrawlerStatus startCrawl()

Description copied from class: PluggableCrawl

Enqueue a crawl request.

Specified by:

startCrawl in class PluggableCrawl

Returns:

the crawler status

stopCrawl

public org.lockss.crawler.CrawlerStatus stopCrawl()

Description copied from class: PluggableCrawl

Stop crawl crawler status.

Specified by:

stopCrawl in class PluggableCrawl

Returns:

the crawler status

getTmpDir

public File getTmpDir()

Gets tmp dir.

Returns:

the tmp dir

getWarcFiles

public Collection<File> getWarcFiles(List<String> exts)

getCommand

public List<String> getCommand()

Gets command.

Returns:

the command

getReqUrls

protected List<String> getReqUrls()

getStems

protected List<String> getStems()

getRunnable

public org.lockss.daemon.LockssRunnable getRunnable()

parseLine

public void parseLine(String pre,
 String line)

extractUrls

public static List<String> extractUrls(String text)

Returns a list with all links contained in the input

extractBytes

public static long extractBytes(String str)

toString

public String toString()

Overrides:

toString in class Object