org.bytedeco.arrow

Class MessageDecoder

java.lang.Object

org.bytedeco.javacpp.Pointer

org.bytedeco.arrow.MessageDecoder

All Implemented Interfaces:

AutoCloseable

@Namespace(value="arrow::ipc")
 @NoOffset
 @Properties(inherit=arrow.class)
public class MessageDecoder
extends Pointer

\class MessageDecoder \brief Push style message decoder that receives data from user. This API is EXPERIMENTAL.

Since:

0.17.0

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MessageDecoder.State \brief State for reading a message

Nested classes/interfaces inherited from class org.bytedeco.javacpp.Pointer

Pointer.CustomDeallocator, Pointer.Deallocator, Pointer.NativeDeallocator, Pointer.ReferenceCounter

Field Summary

Fields inherited from class org.bytedeco.javacpp.Pointer

address, capacity, limit, position

Constructor Summary

Constructors

Constructor and Description
MessageDecoder(MessageDecoderListener listener)
MessageDecoder(MessageDecoderListener listener, int initial_state, long initial_next_required_size)
MessageDecoder(MessageDecoderListener listener, int initial_state, long initial_next_required_size, MemoryPool pool)
MessageDecoder(MessageDecoderListener listener, MemoryPool pool) \brief Construct a message decoder.
MessageDecoder(MessageDecoderListener listener, MessageDecoder.State initial_state, long initial_next_required_size)
MessageDecoder(MessageDecoderListener listener, MessageDecoder.State initial_state, long initial_next_required_size, MemoryPool pool) \brief Construct a message decoder with the specified state.
MessageDecoder(Pointer p) Pointer cast constructor.

Method Summary

Modifier and Type	Method and Description
Status	Consume(ArrowBuffer buffer) \brief Feed data to the decoder as a Buffer.
Status	Consume(byte[] data, long size)
Status	Consume(ByteBuffer data, long size)
Status	Consume(BytePointer data, long size) \brief Feed data to the decoder as a raw data.
long	next_required_size() \brief Return the number of bytes needed to advance the state of the decoder.
MessageDecoder.State	state() \brief Return the current state of the decoder.

Methods inherited from class org.bytedeco.javacpp.Pointer

address, asBuffer, asByteBuffer, availablePhysicalBytes, calloc, capacity, capacity, close, deallocate, deallocate, deallocateReferences, deallocator, deallocator, equals, fill, formatBytes, free, getPointer, getPointer, getPointer, getPointer, hashCode, isNull, isNull, limit, limit, malloc, maxBytes, maxPhysicalBytes, memchr, memcmp, memcpy, memmove, memset, offsetAddress, offsetof, offsetof, parseBytes, physicalBytes, position, position, put, realloc, referenceCount, releaseReference, retainReference, setNull, sizeof, sizeof, toString, totalBytes, totalCount, totalPhysicalBytes, withDeallocator, zero

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

MessageDecoder

public MessageDecoder(Pointer p)

Pointer cast constructor. Invokes Pointer(Pointer).

MessageDecoder

public MessageDecoder(@SharedPtr
                      MessageDecoderListener listener,
                      MemoryPool pool)

\brief Construct a message decoder.

Parameters:

listener - [in] a MessageDecoderListener that responds events from the decoder

pool - [in] an optional MemoryPool to copy metadata on the CPU, if required

MessageDecoder

public MessageDecoder(@SharedPtr
                      MessageDecoderListener listener)

MessageDecoder

public MessageDecoder(@SharedPtr
                      MessageDecoderListener listener,
                      MessageDecoder.State initial_state,
                      @Cast(value="int64_t")
                      long initial_next_required_size,
                      MemoryPool pool)

\brief Construct a message decoder with the specified state. This is a construct for advanced users that know how to decode Message.

Parameters:

listener - [in] a MessageDecoderListener that responds events from the decoder

initial_state - [in] an initial state of the decode

initial_next_required_size - [in] the number of bytes needed to run the next action

pool - [in] an optional MemoryPool to copy metadata on the CPU, if required

MessageDecoder

public MessageDecoder(@SharedPtr
                      MessageDecoderListener listener,
                      MessageDecoder.State initial_state,
                      @Cast(value="int64_t")
                      long initial_next_required_size)

MessageDecoder

public MessageDecoder(@SharedPtr
                      MessageDecoderListener listener,
                      @Cast(value="arrow::ipc::MessageDecoder::State")
                      int initial_state,
                      @Cast(value="int64_t")
                      long initial_next_required_size,
                      MemoryPool pool)

MessageDecoder

public MessageDecoder(@SharedPtr
                      MessageDecoderListener listener,
                      @Cast(value="arrow::ipc::MessageDecoder::State")
                      int initial_state,
                      @Cast(value="int64_t")
                      long initial_next_required_size)

Method Detail

Consume

@ByVal
public Status Consume(@Cast(value="const uint8_t*")
                             BytePointer data,
                             @Cast(value="int64_t")
                             long size)

\brief Feed data to the decoder as a raw data. If the decoder can decode one or more messages by the data, the decoder calls listener->OnMessageDecoded() with a decoded message multiple times. If the state of the decoder is changed, corresponding callbacks on listener is called: * MessageDecoder::State::INITIAL: listener->OnInitial() * MessageDecoder::State::METADATA_LENGTH: listener->OnMetadataLength() * MessageDecoder::State::METADATA: listener->OnMetadata() * MessageDecoder::State::BODY: listener->OnBody() * MessageDecoder::State::EOS: listener->OnEOS()

Parameters:

data - [in] a raw data to be processed. This data isn't copied. The passed memory must be kept alive through message processing.

size - [in] raw data size.

Returns:

Status

Consume

@ByVal
public Status Consume(@Cast(value="const uint8_t*")
                             ByteBuffer data,
                             @Cast(value="int64_t")
                             long size)

Consume

@ByVal
public Status Consume(@Cast(value="const uint8_t*")
                             byte[] data,
                             @Cast(value="int64_t")
                             long size)

Consume

@ByVal
public Status Consume(@SharedPtr
                             ArrowBuffer buffer)

\brief Feed data to the decoder as a Buffer. If the decoder can decode one or more messages by the Buffer, the decoder calls listener->OnMessageDecoded() with a decoded message multiple times.

Parameters:

buffer - [in] a Buffer to be processed.

Returns:

Status

next_required_size

@Cast(value="int64_t")
public long next_required_size()

\brief Return the number of bytes needed to advance the state of the decoder. This method is provided for users who want to optimize performance. Normal users don't need to use this method. Here is an example usage for normal users: ~~~{.cpp} decoder.Consume(buffer1); decoder.Consume(buffer2); decoder.Consume(buffer3); ~~~ Decoder has internal buffer. If consumed data isn't enough to advance the state of the decoder, consumed data is buffered to the internal buffer. It causes performance overhead. If you pass next_required_size() size data to each Consume() call, the decoder doesn't use its internal buffer. It improves performance. Here is an example usage to avoid using internal buffer: ~~~{.cpp} buffer1 = get_data(decoder.next_required_size()); decoder.Consume(buffer1); buffer2 = get_data(decoder.next_required_size()); decoder.Consume(buffer2); ~~~ Users can use this method to avoid creating small chunks. Message body must be contiguous data. If users pass small chunks to the decoder, the decoder needs concatenate small chunks internally. It causes performance overhead. Here is an example usage to reduce small chunks: ~~~{.cpp} buffer = AllocateResizableBuffer(); while ((small_chunk = get_data(&small_chunk_size))) { auto current_buffer_size = buffer->size(); buffer->Resize(current_buffer_size + small_chunk_size); memcpy(buffer->mutable_data() + current_buffer_size, small_chunk, small_chunk_size); if (buffer->size() < decoder.next_required_size()) { continue; } std::shared_ptr chunk(buffer.release()); decoder.Consume(chunk); buffer = AllocateResizableBuffer(); } if (buffer->size() > 0) { std::shared_ptr chunk(buffer.release()); decoder.Consume(chunk); } ~~~

Returns:

the number of bytes needed to advance the state of the decoder

state

public MessageDecoder.State state()

\brief Return the current state of the decoder. This method is provided for users who want to optimize performance. Normal users don't need to use this method. Decoder doesn't need Buffer to process data on the MessageDecoder::State::INITIAL state and the MessageDecoder::State::METADATA_LENGTH. Creating Buffer has performance overhead. Advanced users can avoid creating Buffer by checking the current state of the decoder: ~~~{.cpp} switch (decoder.state()) { MessageDecoder::State::INITIAL: MessageDecoder::State::METADATA_LENGTH: { uint8_t data[sizeof(int32_t)]; auto data_size = input->Read(decoder.next_required_size(), data); decoder.Consume(data, data_size); } break; default: { auto buffer = input->Read(decoder.next_required_size()); decoder.Consume(buffer); } break; } ~~~

Returns:

the current state