ai.preferred.venom.fetcher

Class HttpFetcher

java.lang.Object

ai.preferred.venom.fetcher.AbstractClosableFetcher

ai.preferred.venom.fetcher.HttpFetcher

All Implemented Interfaces:

Fetcher, AutoCloseable

public class HttpFetcher
extends AbstractClosableFetcher

This class holds the implementation to provide how items are fetched, to fetch the item, to validate the item and to store it if specified.

Author:

Maksim Tkachenko, Truong Quoc Tuan, Ween Jiann Lee

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	HttpFetcher.Builder

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	HttpFetcher(HttpFetcher.Builder builder)

Method Summary

Modifier and Type	Method and Description
static HttpFetcher	buildDefault()
static HttpFetcher.Builder	builder()
void	close()
Future<Response>	fetch(Request request) Fetch the desired HTTP page given in Request.
Future<Response>	fetch(Request request, Crawler.CallbackProcessor callback) Fetch the desired HTTP page given in Request.
boolean	isRunning()
void	start()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HttpFetcher

protected HttpFetcher(HttpFetcher.Builder builder)

Method Detail

buildDefault

public static HttpFetcher buildDefault()

builder

public static HttpFetcher.Builder builder()

fetch

public Future<Response> fetch(Request request)

Description copied from interface: Fetcher

Fetch the desired HTTP page given in Request.

Parameters:

request - information for the page to fetch.

Returns:

Reponse future

fetch

public Future<Response> fetch(Request request,
                              Crawler.CallbackProcessor callback)

Description copied from interface: Fetcher

Fetch the desired HTTP page given in Request. Executes callback upon completion.

Parameters:

request - information for the page to fetch.

callback - callback

Returns:

Reponse future

isRunning

public boolean isRunning()

Specified by:

isRunning in class AbstractClosableFetcher

start

public void start()

Specified by:

start in class AbstractClosableFetcher

close

public void close()
           throws Exception

Throws:

Exception