ai.preferred.venom.fetcher

Class HttpCloseableFetcher

java.lang.Object

ai.preferred.venom.fetcher.AbstractCloseableFetcher

ai.preferred.venom.fetcher.HttpCloseableFetcher

All Implemented Interfaces:

Fetcher, AutoCloseable

public class HttpCloseableFetcher
extends AbstractCloseableFetcher

This class holds the implementation to provide how items are fetched, to fetch the item, to validate the item and to store it if specified.

Author:

Maksim Tkachenko, Truong Quoc Tuan, Ween Jiann Lee

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	HttpCloseableFetcher.Builder

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	HttpCloseableFetcher(HttpCloseableFetcher.Builder builder)

Method Summary

Modifier and Type	Method and Description
static HttpCloseableFetcher	buildDefault()
static HttpCloseableFetcher.Builder	builder()
void	close()
Future<Response>	fetch(Request request) Fetch the desired HTTP page given in Request.
Future<Response>	fetch(Request request, Crawler.CallbackProcessor callback) Fetch the desired HTTP page given in Request.
void	start()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HttpCloseableFetcher

protected HttpCloseableFetcher(HttpCloseableFetcher.Builder builder)

Method Detail

buildDefault

public static HttpCloseableFetcher buildDefault()

builder

public static HttpCloseableFetcher.Builder builder()

fetch

public Future<Response> fetch(Request request)

Description copied from interface: Fetcher

Fetch the desired HTTP page given in Request.

Parameters:

request - information for the page to fetch.

Returns:

Response future

fetch

public Future<Response> fetch(Request request,
                              Crawler.CallbackProcessor callback)

Description copied from interface: Fetcher

Fetch the desired HTTP page given in Request. Executes callback upon completion.

Parameters:

request - information for the page to fetch.

callback - callback

Returns:

Response future

start

public void start()

Specified by:

start in class AbstractCloseableFetcher

close

public void close()
           throws Exception

Throws:

Exception