| Package | Description |
|---|---|
| cn.wanghaomiao.seimi.core | |
| cn.wanghaomiao.seimi.def | |
| cn.wanghaomiao.seimi.struct |
| Modifier and Type | Method and Description |
|---|---|
Request |
SeimiQueue.bPop(String crawlerName)
阻塞式出队一个请求
|
| Modifier and Type | Method and Description |
|---|---|
void |
SeimiQueue.addProcessed(Request req)
记录一个处理过的请求
|
void |
SeimiCrawler.handleErrorRequest(Request request)
当一个请求处理异常次数超过开发者所设置或是默认设置的最大重新处理次数时会调用该方法记录异常请求
|
boolean |
SeimiQueue.isProcessed(Request req)
判断一个URL是否处理过了
|
boolean |
SeimiQueue.push(Request req)
入队一个请求
|
| Modifier and Type | Method and Description |
|---|---|
Request |
DefaultLocalQueue.bPop(String crawlerName) |
Request |
DefaultRedisQueue.bPop(String crawlerName) |
| Modifier and Type | Method and Description |
|---|---|
LinkedBlockingQueue<Request> |
DefaultLocalQueue.getQueue(String crawlerName) |
| Modifier and Type | Method and Description |
|---|---|
void |
DefaultLocalQueue.addProcessed(Request req) |
void |
DefaultRedisQueue.addProcessed(Request req) |
void |
BaseSeimiCrawler.handleErrorRequest(Request request) |
boolean |
DefaultLocalQueue.isProcessed(Request req) |
boolean |
DefaultRedisQueue.isProcessed(Request req) |
boolean |
DefaultLocalQueue.push(Request req) |
boolean |
DefaultRedisQueue.push(Request req) |
protected void |
BaseSeimiCrawler.push(Request request) |
| Modifier and Type | Method and Description |
|---|---|
static Request |
Request.build(String url,
String callBack) |
static Request |
Request.build(String url,
String callBack,
HttpMethod httpMethod,
Map<String,String> params,
Map<String,Object> meta) |
static Request |
Request.build(String url,
String callBack,
HttpMethod httpMethod,
Map<String,String> params,
Map<String,Object> meta,
int maxReqcount) |
static Request |
Request.build(String url,
String callBack,
int maxReqCount) |
Request |
Response.getRequest() |
| Modifier and Type | Method and Description |
|---|---|
void |
Response.setRequest(Request request) |
Copyright © 2015. All Rights Reserved.