org.bytedeco.arrow

Class StreamingReader

java.lang.Object

org.bytedeco.javacpp.Pointer

org.bytedeco.arrow.RecordBatchReader

org.bytedeco.arrow.StreamingReader

All Implemented Interfaces:

AutoCloseable

@Namespace(value="arrow::csv")
 @Properties(inherit=arrow.class)
public class StreamingReader
extends RecordBatchReader

\brief A class that reads a CSV file incrementally Caveats: - For now, this is always single-threaded (regardless of ReadOptions::use_threads. - Type inference is done on the first block and types are frozen afterwards; to make sure the right data types are inferred, either set ReadOptions::block_size to a large enough value, or use ConvertOptions::column_types to set the desired data types explicitly.

Nested Class Summary

Nested classes/interfaces inherited from class org.bytedeco.javacpp.Pointer

Pointer.CustomDeallocator, Pointer.Deallocator, Pointer.NativeDeallocator, Pointer.ReferenceCounter

Field Summary

Fields inherited from class org.bytedeco.javacpp.Pointer

address, capacity, limit, position

Constructor Summary

Constructors

Constructor and Description
StreamingReader(Pointer p) Pointer cast constructor.

Method Summary

Modifier and Type	Method and Description
long	bytes_read() \brief Return the number of bytes which have been read and processed The returned number includes CSV bytes which the StreamingReader has finished processing, but not bytes for which some processing (e.g.
static StreamingReaderResult	Make(IOContext io_context, InputStream input, ReadOptions arg2, CsvParseOptions arg3, ConvertOptions arg4)
static StreamingReaderResult	Make(MemoryPool pool, InputStream input, ReadOptions read_options, CsvParseOptions parse_options, ConvertOptions convert_options) Deprecated.
static StreamingReaderFuture	MakeAsync(IOContext io_context, InputStream input, Executor cpu_executor, ReadOptions arg3, CsvParseOptions arg4, ConvertOptions arg5) Create a StreamingReader instance This involves some I/O as the first batch must be loaded during the creation process so it is returned as a future Currently, the StreamingReader is not async-reentrant and does not do any fan-out parsing (see ARROW-11889)
RecordBatchFuture	ReadNextAsync()

Methods inherited from class org.bytedeco.arrow.RecordBatchReader

Make, Make, Next, ReadAll, ReadAll, ReadNext, schema

Methods inherited from class org.bytedeco.javacpp.Pointer

address, asBuffer, asByteBuffer, availablePhysicalBytes, calloc, capacity, capacity, close, deallocate, deallocate, deallocateReferences, deallocator, deallocator, equals, fill, formatBytes, free, getPointer, getPointer, getPointer, getPointer, hashCode, interruptDeallocatorThread, isNull, isNull, limit, limit, malloc, maxBytes, maxPhysicalBytes, memchr, memcmp, memcpy, memmove, memset, offsetAddress, offsetof, offsetof, parseBytes, physicalBytes, position, position, put, realloc, referenceCount, releaseReference, retainReference, setNull, sizeof, sizeof, toString, totalBytes, totalCount, totalPhysicalBytes, withDeallocator, zero

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

StreamingReader

public StreamingReader(Pointer p)

Pointer cast constructor. Invokes Pointer(Pointer).

Method Detail

ReadNextAsync

@ByVal
public RecordBatchFuture ReadNextAsync()

bytes_read

@Cast(value="int64_t")
public long bytes_read()

\brief Return the number of bytes which have been read and processed The returned number includes CSV bytes which the StreamingReader has finished processing, but not bytes for which some processing (e.g. CSV parsing or conversion to Arrow layout) is still ongoing. Furthermore, the following rules apply: - bytes skipped by ReadOptions.skip_rows are counted as being read before any records are returned. - bytes read while parsing the header are counted as being read before any records are returned. - bytes skipped by ReadOptions.skip_rows_after_names are counted after the first batch is returned.

MakeAsync

@ByVal
public static StreamingReaderFuture MakeAsync(@ByVal
                                                      IOContext io_context,
                                                      @SharedPtr @Cast(value={"","std::shared_ptr<arrow::io::InputStream>"})
                                                      InputStream input,
                                                      Executor cpu_executor,
                                                      @Const @ByRef
                                                      ReadOptions arg3,
                                                      @Const @ByRef
                                                      CsvParseOptions arg4,
                                                      @Const @ByRef
                                                      ConvertOptions arg5)

Create a StreamingReader instance This involves some I/O as the first batch must be loaded during the creation process so it is returned as a future Currently, the StreamingReader is not async-reentrant and does not do any fan-out parsing (see ARROW-11889)

Make

@ByVal
public static StreamingReaderResult Make(@ByVal
                                                 IOContext io_context,
                                                 @SharedPtr @Cast(value={"","std::shared_ptr<arrow::io::InputStream>"})
                                                 InputStream input,
                                                 @Const @ByRef
                                                 ReadOptions arg2,
                                                 @Const @ByRef
                                                 CsvParseOptions arg3,
                                                 @Const @ByRef
                                                 ConvertOptions arg4)

Make

@Deprecated
 @ByVal
public static StreamingReaderResult Make(MemoryPool pool,
                                                               @SharedPtr @Cast(value={"","std::shared_ptr<arrow::io::InputStream>"})
                                                               InputStream input,
                                                               @Const @ByRef
                                                               ReadOptions read_options,
                                                               @Const @ByRef
                                                               CsvParseOptions parse_options,
                                                               @Const @ByRef
                                                               ConvertOptions convert_options)

Deprecated.