| Package | Description |
|---|---|
| ai.preferred.venom.fetcher |
| Modifier and Type | Method and Description |
|---|---|
static AsyncFetcher.Builder |
AsyncFetcher.builder()
Create an instance of builder.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.disableCompression()
Disables request for compress pages and to decompress pages
after it is fetched.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.disableCookies()
Disables cookie storage.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.register(@NotNull Callback callback)
Register any callbacks that will be called when a page has been fetched.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setConnectionRequestTimeout(int connectionRequestTimeout)
The timeout in milliseconds used when requesting a connection
from the connection manager.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setConnectTimeout(int connectTimeout)
Determines the timeout in milliseconds until a connection is established.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setFileManager(@NotNull FileManager fileManager)
Sets the FileManager to be used.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setHeaders(@NotNull Map<String,String> headers)
Sets the headers to be used when fetching items.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setMaxConnections(int maxConnections)
Sets the maximum allowable connections at an instance.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setMaxRouteConnections(int maxRouteConnections)
Sets the maximum allowable connections at an instance for
a particular route (host).
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setNumIoThreads(int numIoThreads)
Number of httpclient dispatcher threads.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setProxyProvider(@NotNull ProxyProvider proxyProvider)
Sets the ProxyProvider to be used.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setRedirectStrategy(org.apache.http.client.RedirectStrategy redirectStrategy)
Sets the redirection strategy for a response received by the fetcher.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setSocketTimeout(int socketTimeout)
Defines the socket timeout (
SO_TIMEOUT) in milliseconds,
which is the timeout for waiting for data or, put differently,
a maximum period inactivity between two consecutive data packets). |
AsyncFetcher.Builder |
AsyncFetcher.Builder.setSslContext(SSLContext sslContext)
Sets the ssl context for an encrypted response.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setStopCodes(int... codes)
Set a list of stop code that will interrupt crawling.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setThreadFactory(@NotNull ThreadFactory threadFactory)
Set the thread factory that creates the httpclient dispatcher
threads.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setUserAgent(@NotNull UserAgent userAgent)
Sets the UserAgent to be used, if not set, default will be chosen.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setValidator(Validator... validators)
Sets the multiple validators to be used.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setValidator(@NotNull Validator validator)
Sets the Validator to be used.
|
AsyncFetcher.Builder |
AsyncFetcher.Builder.setValidatorRouter(@NotNull ValidatorRouter router)
Sets ValidatorRouter to be used.
|
Copyright © 2019. All rights reserved.