A B C D G H I L N P Q R S T V W X

A

add(String, HttpRequestProcessor) - Method in class cn.wanghaomiao.seimi.httpd.SeimiHttpHandler

after(Method, Response) - Method in interface cn.wanghaomiao.seimi.core.SeimiInterceptor

可以在目标方法执行之后定义一些处理逻辑

allowRules() - Method in interface cn.wanghaomiao.seimi.core.SeimiCrawler

用于设置允许的请求URL匹配规则

allowRules() - Method in class cn.wanghaomiao.seimi.def.BaseSeimiCrawler

applicationContext - Variable in class cn.wanghaomiao.seimi.core.SeimiContext

B

BaseSeimiCrawler - Class in cn.wanghaomiao.seimi.def

BaseSeimiCrawler() - Constructor for class cn.wanghaomiao.seimi.def.BaseSeimiCrawler

before(Method, Response) - Method in interface cn.wanghaomiao.seimi.core.SeimiInterceptor

可以在目标方法执行之前定义一些处理逻辑

BodyType - Enum in cn.wanghaomiao.seimi.struct

bPop(String) - Method in interface cn.wanghaomiao.seimi.core.SeimiQueue

阻塞式出队一个请求

bPop(String) - Method in class cn.wanghaomiao.seimi.def.DefaultLocalQueue

bPop(String) - Method in class cn.wanghaomiao.seimi.def.DefaultRedisQueue

build(String, String, HttpMethod, Map<String, String>, Map<String, Object>) - Static method in class cn.wanghaomiao.seimi.struct.Request

build(String, String) - Static method in class cn.wanghaomiao.seimi.struct.Request

C

castTo(String) - Method in interface cn.wanghaomiao.seimi.core.CastToNumber

CastToNumber - Interface in cn.wanghaomiao.seimi.core

String转数字接口

castToNumber(Class, String) - Static method in class cn.wanghaomiao.seimi.utils.GenericUtils

cliBuilder(int) - Static method in class cn.wanghaomiao.seimi.http.HttpClientFactory

cn.wanghaomiao.seimi.annotation - package cn.wanghaomiao.seimi.annotation

cn.wanghaomiao.seimi.annotation.validate - package cn.wanghaomiao.seimi.annotation.validate

cn.wanghaomiao.seimi.core - package cn.wanghaomiao.seimi.core

cn.wanghaomiao.seimi.def - package cn.wanghaomiao.seimi.def

cn.wanghaomiao.seimi.exception - package cn.wanghaomiao.seimi.exception

cn.wanghaomiao.seimi.http - package cn.wanghaomiao.seimi.http

cn.wanghaomiao.seimi.httpd - package cn.wanghaomiao.seimi.httpd

cn.wanghaomiao.seimi.struct - package cn.wanghaomiao.seimi.struct

cn.wanghaomiao.seimi.utils - package cn.wanghaomiao.seimi.utils

CommonObject - Class in cn.wanghaomiao.seimi.struct

CommonObject() - Constructor for class cn.wanghaomiao.seimi.struct.CommonObject

cookieStore - Variable in class cn.wanghaomiao.seimi.def.BaseSeimiCrawler

Crawler - Annotation Type in cn.wanghaomiao.seimi.annotation

用于定义一个类为爬虫规则文件

CrawlerModel - Class in cn.wanghaomiao.seimi.struct

CrawlerModel(Class<? extends BaseSeimiCrawler>, ApplicationContext) - Constructor for class cn.wanghaomiao.seimi.struct.CrawlerModel

crawlerModelContext - Variable in class cn.wanghaomiao.seimi.core.SeimiContext

crawlerName - Variable in class cn.wanghaomiao.seimi.def.BaseSeimiCrawler

crawlerName - Variable in class cn.wanghaomiao.seimi.httpd.HttpRequestProcessor

crawlers - Variable in class cn.wanghaomiao.seimi.core.SeimiContext

CrawlerStatusHttpProcessor - Class in cn.wanghaomiao.seimi.httpd

CrawlerStatusHttpProcessor(SeimiQueue, String) - Constructor for class cn.wanghaomiao.seimi.httpd.CrawlerStatusHttpProcessor

D

DefaultLocalQueue - Class in cn.wanghaomiao.seimi.def

DefaultLocalQueue() - Constructor for class cn.wanghaomiao.seimi.def.DefaultLocalQueue

DefaultRedisQueue - Class in cn.wanghaomiao.seimi.def

DefaultRedisQueue() - Constructor for class cn.wanghaomiao.seimi.def.DefaultRedisQueue

denyRules() - Method in interface cn.wanghaomiao.seimi.core.SeimiCrawler

用于设置要放弃访问的请求URL匹配规则

denyRules() - Method in class cn.wanghaomiao.seimi.def.BaseSeimiCrawler

document() - Method in class cn.wanghaomiao.seimi.struct.Response

G

GenericUtils - Class in cn.wanghaomiao.seimi.utils

实现工具类，检查参数化类型的参数类型。

GenericUtils() - Constructor for class cn.wanghaomiao.seimi.utils.GenericUtils

getActualClass(Type) - Static method in class cn.wanghaomiao.seimi.utils.GenericUtils

从参数, 返回值, 基类的: Generic 类型信息获取传入的实际类信息。

getBodyType() - Method in class cn.wanghaomiao.seimi.struct.Response

getCallBack() - Method in class cn.wanghaomiao.seimi.struct.Request

getCharset() - Method in class cn.wanghaomiao.seimi.struct.Response

getClazz() - Method in class cn.wanghaomiao.seimi.struct.CrawlerModel

getContent() - Method in class cn.wanghaomiao.seimi.struct.Response

getContext() - Method in class cn.wanghaomiao.seimi.core.SeimiScanner

getContext() - Method in class cn.wanghaomiao.seimi.struct.CrawlerModel

getCookieStore() - Method in interface cn.wanghaomiao.seimi.core.SeimiCrawler

如果开启cookies通过此方法获取cookiesStore

getCookieStore() - Method in class cn.wanghaomiao.seimi.def.BaseSeimiCrawler

getCrawlerName() - Method in class cn.wanghaomiao.seimi.def.BaseSeimiCrawler

getCrawlerName() - Method in class cn.wanghaomiao.seimi.struct.CrawlerModel

getCrawlerName() - Method in class cn.wanghaomiao.seimi.struct.Request

getData() - Method in class cn.wanghaomiao.seimi.struct.Response

getDelay() - Method in class cn.wanghaomiao.seimi.struct.CrawlerModel

getHcPoolInstance() - Static method in class cn.wanghaomiao.seimi.http.HttpClientConnectionManagerProvider

getHost() - Method in class cn.wanghaomiao.seimi.def.DefaultRedisQueue

getHttpClient() - Static method in class cn.wanghaomiao.seimi.http.HttpClientFactory

getHttpClient(int) - Static method in class cn.wanghaomiao.seimi.http.HttpClientFactory

getHttpClient(int, CookieStore) - Static method in class cn.wanghaomiao.seimi.http.HttpClientFactory

getHttpMethod() - Method in class cn.wanghaomiao.seimi.struct.Request

getHttpResponse() - Method in class cn.wanghaomiao.seimi.struct.Response

getInstance() - Method in class cn.wanghaomiao.seimi.struct.CrawlerModel

getMemberMethods() - Method in class cn.wanghaomiao.seimi.struct.CrawlerModel

getMeta() - Method in class cn.wanghaomiao.seimi.struct.Request

getMeta() - Method in class cn.wanghaomiao.seimi.struct.Response

getParams() - Method in class cn.wanghaomiao.seimi.struct.Request

getParams() - Method in class cn.wanghaomiao.seimi.struct.Response

getPassword() - Method in class cn.wanghaomiao.seimi.def.DefaultRedisQueue

getPort() - Method in class cn.wanghaomiao.seimi.def.DefaultRedisQueue

getProcessedSet(String) - Method in class cn.wanghaomiao.seimi.def.DefaultLocalQueue

getProxy() - Method in class cn.wanghaomiao.seimi.struct.CrawlerModel

getQueue(String) - Method in class cn.wanghaomiao.seimi.def.DefaultLocalQueue

getqueueClass() - Method in class cn.wanghaomiao.seimi.struct.CrawlerModel

getQueueInstance() - Method in class cn.wanghaomiao.seimi.struct.CrawlerModel

getQuueNamePrefix() - Method in class cn.wanghaomiao.seimi.def.DefaultRedisQueue

getRealUrl() - Method in class cn.wanghaomiao.seimi.struct.Response

getRedirect(HttpRequest, HttpResponse, HttpContext) - Method in class cn.wanghaomiao.seimi.http.SeimiRedirectStrategy

getReferer() - Method in class cn.wanghaomiao.seimi.struct.Response

getReponseEntity() - Method in class cn.wanghaomiao.seimi.struct.Response

getRequest() - Method in class cn.wanghaomiao.seimi.struct.Response

getTargetAnnotationClass() - Method in interface cn.wanghaomiao.seimi.core.SeimiInterceptor

获取目标方法应标记的注解

getUrl() - Method in class cn.wanghaomiao.seimi.struct.Request

getUrl() - Method in class cn.wanghaomiao.seimi.struct.Response

getUserAgent() - Method in interface cn.wanghaomiao.seimi.core.SeimiCrawler

getUserAgent() - Method in class cn.wanghaomiao.seimi.def.BaseSeimiCrawler

getWClient() - Method in class cn.wanghaomiao.seimi.def.DefaultRedisQueue

getWeight() - Method in interface cn.wanghaomiao.seimi.core.SeimiInterceptor

当需要控制多个拦截器执行的先后顺序时可以重写这个方法

getWritePool() - Method in class cn.wanghaomiao.seimi.def.DefaultRedisQueue

H

handle(String, Request, HttpServletRequest, HttpServletResponse) - Method in class cn.wanghaomiao.seimi.httpd.SeimiHttpHandler

handleHttpRequest(HttpServletRequest, HttpServletResponse) - Method in class cn.wanghaomiao.seimi.httpd.CrawlerStatusHttpProcessor

handleHttpRequest(HttpServletRequest, HttpServletResponse) - Method in class cn.wanghaomiao.seimi.httpd.HttpRequestProcessor

handleHttpRequest(HttpServletRequest, HttpServletResponse) - Method in class cn.wanghaomiao.seimi.httpd.PushRequestHttpProcessor

HttpClientConnectionManagerProvider - Class in cn.wanghaomiao.seimi.http

HttpClientConnectionManagerProvider() - Constructor for class cn.wanghaomiao.seimi.http.HttpClientConnectionManagerProvider

HttpClientFactory - Class in cn.wanghaomiao.seimi.http

HttpClientFactory() - Constructor for class cn.wanghaomiao.seimi.http.HttpClientFactory

HttpMethod - Enum in cn.wanghaomiao.seimi.http

HttpRequestProcessor - Class in cn.wanghaomiao.seimi.httpd

HttpRequestProcessor(SeimiQueue, String) - Constructor for class cn.wanghaomiao.seimi.httpd.HttpRequestProcessor

I

info(String, Object...) - Static method in class cn.wanghaomiao.seimi.utils.StrFormatUtil

init() - Method in class cn.wanghaomiao.seimi.def.DefaultRedisQueue

Interceptor - Annotation Type in cn.wanghaomiao.seimi.annotation

标记一个拦截器，用于解析引擎识别

interceptors - Variable in class cn.wanghaomiao.seimi.core.SeimiContext

isNumber(Class) - Static method in class cn.wanghaomiao.seimi.utils.GenericUtils

判断给定类是否是支持的数字类型

isProcessed(Request) - Method in interface cn.wanghaomiao.seimi.core.SeimiQueue

判断一个URL是否处理过了

isProcessed(Request) - Method in class cn.wanghaomiao.seimi.def.DefaultLocalQueue

isProcessed(Request) - Method in class cn.wanghaomiao.seimi.def.DefaultRedisQueue

isStop() - Method in class cn.wanghaomiao.seimi.struct.Request

isUseCookie() - Method in class cn.wanghaomiao.seimi.struct.CrawlerModel

isUseUnrepeated() - Method in class cn.wanghaomiao.seimi.struct.CrawlerModel

L

len(String) - Method in interface cn.wanghaomiao.seimi.core.SeimiQueue

任务队列剩余长度

len(String) - Method in class cn.wanghaomiao.seimi.def.DefaultLocalQueue

len(String) - Method in class cn.wanghaomiao.seimi.def.DefaultRedisQueue

logger - Variable in class cn.wanghaomiao.seimi.core.SeimiContext

logger - Variable in class cn.wanghaomiao.seimi.def.BaseSeimiCrawler

N

NotNull - Annotation Type in cn.wanghaomiao.seimi.annotation.validate

用于校验一个字段是否为空

P

parse(Class<T>, String) - Static method in class cn.wanghaomiao.seimi.core.SeimiBeanResolver

proxy() - Method in interface cn.wanghaomiao.seimi.core.SeimiCrawler

可以自定义返回随机的代理

proxy() - Method in class cn.wanghaomiao.seimi.def.BaseSeimiCrawler

push(Request) - Method in interface cn.wanghaomiao.seimi.core.SeimiQueue

入队一个请求

push(Request) - Method in class cn.wanghaomiao.seimi.def.BaseSeimiCrawler

push(Request) - Method in class cn.wanghaomiao.seimi.def.DefaultLocalQueue

push(Request) - Method in class cn.wanghaomiao.seimi.def.DefaultRedisQueue

PushRequestHttpProcessor - Class in cn.wanghaomiao.seimi.httpd

PushRequestHttpProcessor(SeimiQueue, String) - Constructor for class cn.wanghaomiao.seimi.httpd.PushRequestHttpProcessor

Q

Queue - Annotation Type in cn.wanghaomiao.seimi.annotation

用来指定一个类为消费队列的实现，只有打上这个注解才能被系统真正识别。

queue - Variable in class cn.wanghaomiao.seimi.def.BaseSeimiCrawler

R

refresh() - Method in class cn.wanghaomiao.seimi.def.DefaultRedisQueue

render(Class<T>) - Method in class cn.wanghaomiao.seimi.struct.Response

通过bean中定义的Xpath注解进行自动填充

Request - Class in cn.wanghaomiao.seimi.struct

封装一个抓取请求的基本信息体

Request(String, String, HttpMethod, Map<String, String>, Map<String, Object>) - Constructor for class cn.wanghaomiao.seimi.struct.Request

Request(String, String) - Constructor for class cn.wanghaomiao.seimi.struct.Request

Request() - Constructor for class cn.wanghaomiao.seimi.struct.Request

Response - Class in cn.wanghaomiao.seimi.struct

Response() - Constructor for class cn.wanghaomiao.seimi.struct.Response

run() - Method in class cn.wanghaomiao.seimi.core.SeimiProcessor

S

saveTo(File) - Method in class cn.wanghaomiao.seimi.struct.Response

scan(String[], Class<? extends Annotation>...) - Method in class cn.wanghaomiao.seimi.core.SeimiScanner

ScanConfig - Class in cn.wanghaomiao.seimi.core

ScanConfig() - Constructor for class cn.wanghaomiao.seimi.core.ScanConfig

Seimi - Class in cn.wanghaomiao.seimi.core

Seimi() - Constructor for class cn.wanghaomiao.seimi.core.Seimi

SeimiBeanResolveException - Exception in cn.wanghaomiao.seimi.exception

SeimiBeanResolveException(String) - Constructor for exception cn.wanghaomiao.seimi.exception.SeimiBeanResolveException

SeimiBeanResolver - Class in cn.wanghaomiao.seimi.core

根据Bean中字段定义的XPath路径自动提取数据

SeimiBeanResolver() - Constructor for class cn.wanghaomiao.seimi.core.SeimiBeanResolver

SeimiContext - Class in cn.wanghaomiao.seimi.core

初始化上下文环境

SeimiContext() - Constructor for class cn.wanghaomiao.seimi.core.SeimiContext

SeimiCrawler - Interface in cn.wanghaomiao.seimi.core

SeimiHttpHandler - Class in cn.wanghaomiao.seimi.httpd

SeimiHttpHandler(Map<String, CrawlerModel>) - Constructor for class cn.wanghaomiao.seimi.httpd.SeimiHttpHandler

SeimiInitExcepiton - Exception in cn.wanghaomiao.seimi.exception

启动初始化异常

SeimiInitExcepiton(String) - Constructor for exception cn.wanghaomiao.seimi.exception.SeimiInitExcepiton

SeimiInterceptor - Interface in cn.wanghaomiao.seimi.core

爬虫执行方法的通用拦截器

SeimiProcessor - Class in cn.wanghaomiao.seimi.core

SeimiProcessor(List<SeimiInterceptor>, CrawlerModel) - Constructor for class cn.wanghaomiao.seimi.core.SeimiProcessor

SeimiQueue - Interface in cn.wanghaomiao.seimi.core

定义系统队列的基本接口，可自由选择实现，只要符合规范就行。

seimiQueue - Variable in class cn.wanghaomiao.seimi.httpd.HttpRequestProcessor

SeimiRedirectStrategy - Class in cn.wanghaomiao.seimi.http

之所以要自行覆盖默认实现，是因为默认实现在post/redirect/post这种情况下不会传递原有请求的数据信息，只会传递一个uri其他的都丢了， 这显然是非常不理想的，所以必须重写覆盖。结果还是很不错的。

SeimiRedirectStrategy() - Constructor for class cn.wanghaomiao.seimi.http.SeimiRedirectStrategy

seimiScanner - Variable in class cn.wanghaomiao.seimi.core.SeimiContext

SeimiScanner - Class in cn.wanghaomiao.seimi.core

上下文加载器

SeimiScanner() - Constructor for class cn.wanghaomiao.seimi.core.SeimiScanner

setBodyType(BodyType) - Method in class cn.wanghaomiao.seimi.struct.Response

setCallBack(String) - Method in class cn.wanghaomiao.seimi.struct.Request

setCharset(String) - Method in class cn.wanghaomiao.seimi.struct.Response

setClazz(Class<? extends BaseSeimiCrawler>) - Method in class cn.wanghaomiao.seimi.struct.CrawlerModel

setContent(String) - Method in class cn.wanghaomiao.seimi.struct.Response

setContext(ApplicationContext) - Method in class cn.wanghaomiao.seimi.struct.CrawlerModel

setCrawlerName(String) - Method in class cn.wanghaomiao.seimi.def.BaseSeimiCrawler

setCrawlerName(String) - Method in class cn.wanghaomiao.seimi.struct.Request

setData(byte[]) - Method in class cn.wanghaomiao.seimi.struct.Response

setHost(String) - Method in class cn.wanghaomiao.seimi.def.DefaultRedisQueue

setHttpMethod(HttpMethod) - Method in class cn.wanghaomiao.seimi.struct.Request

setHttpResponse(HttpResponse) - Method in class cn.wanghaomiao.seimi.struct.Response

setInstance(BaseSeimiCrawler) - Method in class cn.wanghaomiao.seimi.struct.CrawlerModel

setMemberMethods(Map<String, Method>) - Method in class cn.wanghaomiao.seimi.struct.CrawlerModel

setMeta(Map<String, Object>) - Method in class cn.wanghaomiao.seimi.struct.Request

setMeta(Map<String, Object>) - Method in class cn.wanghaomiao.seimi.struct.Response

setParams(Map<String, String>) - Method in class cn.wanghaomiao.seimi.struct.Request

setParams(Map<String, String>) - Method in class cn.wanghaomiao.seimi.struct.Response

setPassword(String) - Method in class cn.wanghaomiao.seimi.def.DefaultRedisQueue

setPort(int) - Method in class cn.wanghaomiao.seimi.def.DefaultRedisQueue

setQueue(SeimiQueue) - Method in class cn.wanghaomiao.seimi.def.BaseSeimiCrawler

setqueueClass(Class<? extends SeimiQueue>) - Method in class cn.wanghaomiao.seimi.struct.CrawlerModel

setqueueImpl(SeimiQueue) - Method in class cn.wanghaomiao.seimi.struct.CrawlerModel

setQuueNamePrefix(String) - Method in class cn.wanghaomiao.seimi.def.DefaultRedisQueue

setRealUrl(String) - Method in class cn.wanghaomiao.seimi.struct.Response

setReferer(String) - Method in class cn.wanghaomiao.seimi.struct.Response

setReponseEntity(HttpEntity) - Method in class cn.wanghaomiao.seimi.struct.Response

setRequest(Request) - Method in class cn.wanghaomiao.seimi.struct.Response

setStop(boolean) - Method in class cn.wanghaomiao.seimi.struct.Request

setUrl(String) - Method in class cn.wanghaomiao.seimi.struct.Request

setUrl(String) - Method in class cn.wanghaomiao.seimi.struct.Response

start(String...) - Method in class cn.wanghaomiao.seimi.core.Seimi

主启动 start master

start(Response) - Method in interface cn.wanghaomiao.seimi.core.SeimiCrawler

针对startUrl生成首批的response回调这个初始接口

startAll() - Method in class cn.wanghaomiao.seimi.core.Seimi

startAllWithHttpd(int) - Method in class cn.wanghaomiao.seimi.core.Seimi

startUrls() - Method in interface cn.wanghaomiao.seimi.core.SeimiCrawler

设置起始url

startWithHttpd(int, String...) - Method in class cn.wanghaomiao.seimi.core.Seimi

按名称启动爬虫并开启http服务接口API

startWorkers() - Method in class cn.wanghaomiao.seimi.core.Seimi

StrFormatUtil - Class in cn.wanghaomiao.seimi.utils

StrFormatUtil() - Constructor for class cn.wanghaomiao.seimi.utils.StrFormatUtil

StructValidator - Class in cn.wanghaomiao.seimi.utils

StructValidator() - Constructor for class cn.wanghaomiao.seimi.utils.StructValidator

T

toString() - Method in class cn.wanghaomiao.seimi.struct.CommonObject

totalCrawled(String) - Method in interface cn.wanghaomiao.seimi.core.SeimiQueue

目前总共的抓取数量

totalCrawled(String) - Method in class cn.wanghaomiao.seimi.def.DefaultLocalQueue

totalCrawled(String) - Method in class cn.wanghaomiao.seimi.def.DefaultRedisQueue

V

val() - Method in enum cn.wanghaomiao.seimi.http.HttpMethod

val() - Method in enum cn.wanghaomiao.seimi.struct.BodyType

validateAllowRules(String[], String) - Static method in class cn.wanghaomiao.seimi.utils.StructValidator

validateAnno(Object) - Static method in class cn.wanghaomiao.seimi.utils.StructValidator

validateDenyRules(String[], String) - Static method in class cn.wanghaomiao.seimi.utils.StructValidator

valueOf(String) - Static method in enum cn.wanghaomiao.seimi.http.HttpMethod

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum cn.wanghaomiao.seimi.struct.BodyType

Returns the enum constant of this type with the specified name.

values() - Static method in enum cn.wanghaomiao.seimi.http.HttpMethod

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum cn.wanghaomiao.seimi.struct.BodyType

Returns an array containing the constants of this enum type, in the order they are declared.

W

workersPool - Variable in class cn.wanghaomiao.seimi.core.SeimiContext

X

Xpath - Annotation Type in cn.wanghaomiao.seimi.annotation

定义一个用于提取数据到指定字段的xpath路径

A B C D G H I L N P Q R S T V W X