cn.wanghaomiao.seimi.def

Class DefaultLocalQueue

java.lang.Object

cn.wanghaomiao.seimi.def.DefaultLocalQueue

All Implemented Interfaces:

SeimiQueue

@Queue
public class DefaultLocalQueue
extends Object
implements SeimiQueue

Since:

2015/7/21.

Author:

汪浩淼 et.tw@163.com

Constructor Summary

Constructors

Constructor and Description
DefaultLocalQueue()

Method Summary

Methods

Modifier and Type	Method and Description
void	addProcessed(Request req) 记录一个处理过的请求
Request	bPop(String crawlerName) 阻塞式出队一个请求
ConcurrentSkipListSet<String>	getProcessedSet(String crawlerName)
LinkedBlockingQueue<Request>	getQueue(String crawlerName)
boolean	isProcessed(Request req) 判断一个URL是否处理过了
long	len(String crawlerName) 任务队列剩余长度
boolean	push(Request req) 入队一个请求
long	totalCrawled(String crawlerName) 目前总共的抓取数量

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DefaultLocalQueue

public DefaultLocalQueue()

Method Detail

bPop

public Request bPop(String crawlerName)

Description copied from interface: SeimiQueue

阻塞式出队一个请求

Specified by:

bPop in interface SeimiQueue

Returns:

push

public boolean push(Request req)

Description copied from interface: SeimiQueue

入队一个请求

Specified by:

push in interface SeimiQueue

Returns:

len

public long len(String crawlerName)

Description copied from interface: SeimiQueue

任务队列剩余长度

Specified by:

len in interface SeimiQueue

Returns:

isProcessed

public boolean isProcessed(Request req)

Description copied from interface: SeimiQueue

判断一个URL是否处理过了

Specified by:

isProcessed in interface SeimiQueue

Returns:

addProcessed

public void addProcessed(Request req)

Description copied from interface: SeimiQueue

记录一个处理过的请求

Specified by:

addProcessed in interface SeimiQueue

totalCrawled

public long totalCrawled(String crawlerName)

Description copied from interface: SeimiQueue

目前总共的抓取数量

Specified by:

totalCrawled in interface SeimiQueue

Returns:

getQueue

public LinkedBlockingQueue<Request> getQueue(String crawlerName)

getProcessedSet

public ConcurrentSkipListSet<String> getProcessedSet(String crawlerName)