Package org.pipecraft.pipes.sync.source

Class URLTxtReaderPipe

java.lang.Object

org.pipecraft.pipes.sync.source.InputStreamPipe<String>

org.pipecraft.pipes.sync.source.URLTxtReaderPipe

All Implemented Interfaces:

Closeable, AutoCloseable, BasePipe, Pipe<String>

public class URLTxtReaderPipe
extends InputStreamPipe<String>

A source pipe providing the lines of a text resource defined by a URL. All valid URL scheme types supported by the JDK are supported here, and not just HTTP URLs. For HTTP URLs, server side exceptions are represented as HttpPipeException, which also include the status code. Redirects are supported. Supports decompression.

Author:

Michal Rockban, Eyal Schneider

Constructor Summary

Constructors

Constructor	Description
URLTxtReaderPipe(URL url)	Constructor Uses defaults: UTF8 charset, no compression, and default timeouts.
URLTxtReaderPipe(URL url, Charset charset, FileReadOptions options, int connectTimeoutMs, int readTimeoutMs)	Constructor

Method Summary

Modifier and Type	Method	Description
void	close()
protected SizedInputStream	createInputStream()
String	next()
String	peek()
void	start()	Performs pre-processing prior to item flow throw the pipe.

Methods inherited from class org.pipecraft.pipes.sync.source.InputStreamPipe

getInputStream, getInputStreamSize, getProgress

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

URLTxtReaderPipe

public URLTxtReaderPipe(URL url,
                        Charset charset,
                        FileReadOptions options,
                        int connectTimeoutMs,
                        int readTimeoutMs)

Constructor

Parameters:

url - the URL of the file.

charset - The charset used

options - The file read options

connectTimeoutMs - Connect timeout in milli seconds

readTimeoutMs - Read timeout in millis seconds

URLTxtReaderPipe

public URLTxtReaderPipe(URL url)

Constructor Uses defaults: UTF8 charset, no compression, and default timeouts.

Parameters:

url - the URL of the file.

Method Detail

next

public String next()
            throws PipeException,
                   InterruptedException

Returns:

The next item in this pipe output, or null if the output end has been reached. May be a blocking operation.

Throws:

PipeException - In case of pipe errors in this pipe or somewhere up-stream while trying to prepare next item to return.

InterruptedException - In case that the operation has been interrupted by another thread.

peek

public String peek()

Returns:

The next item in the pipe's output. Does not remove it, so next call to next() will return it.

createInputStream

protected SizedInputStream createInputStream()
                                      throws IOException,
                                             IOPipeException,
                                             InterruptedException

Specified by:

createInputStream in class InputStreamPipe<String>

Returns:

A new input stream based on ctor arguments. Called once in the start() method. Expected to be the raw input stream (without decompression or buffering layers), but should be initialized with expected size (see SizedInputStream). The size should be null if unknown.

Throws:

IOException - (Will be wrapped by IOPipeException)

IOPipeException - (Are propagated without wrapping)

InterruptedException - In case that the operation is interrupted

start

public void start()
           throws PipeException,
                  InterruptedException

Description copied from interface: BasePipe

Performs pre-processing prior to item flow throw the pipe. Implementations must call the same method for all their input pipes before accessing their items. This is typically done here.

Specified by:

start in interface BasePipe

Overrides:

start in class InputStreamPipe<String>

Throws:

PipeException - In case of pipe errors in this pipe or somewhere up-stream.

InterruptedException - In case that the operation has been interrupted by another thread.

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Overrides:

close in class InputStreamPipe<String>

Throws:

IOException