org.bytedeco.tvm

Class InputSplit

java.lang.Object

org.bytedeco.javacpp.Pointer

org.bytedeco.tvm.InputSplit

All Implemented Interfaces:

AutoCloseable

@Namespace(value="dmlc")
 @Properties(inherit=tvm_runtime.class)
public class InputSplit
extends Pointer

\brief input split creates that allows reading of records from split of data, independent part that covers all the dataset see InputSplit::Create for definition of record

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	InputSplit.Blob \brief a blob of memory region

Nested classes/interfaces inherited from class org.bytedeco.javacpp.Pointer

Pointer.CustomDeallocator, Pointer.Deallocator, Pointer.NativeDeallocator, Pointer.ReferenceCounter

Field Summary

Fields inherited from class org.bytedeco.javacpp.Pointer

address, capacity, limit, position

Constructor Summary

Constructors

Constructor and Description
InputSplit(Pointer p) Pointer cast constructor.

Method Summary

Modifier and Type	Method and Description
void	BeforeFirst() \brief reset the position of InputSplit to beginning
long	GetTotalSize() \brief get the total size of the InputSplit
void	HintChunkSize(long chunk_size) \brief hint the inputsplit how large the chunk size it should return when implementing NextChunk this is a hint so may not be enforced, but InputSplit will try adjust its internal buffer size to the hinted value
boolean	NextBatch(InputSplit.Blob out_chunk, long n_records) \brief get a chunk of memory that can contain multiple records, with hint for how many records is needed, the caller needs to parse the content of the resulting chunk, for text file, out_chunk can contain data of multiple lines for recordio, out_chunk can contain multiple records(including headers) This function ensures there won't be partial record in the chunk caller can modify the memory content of out_chunk, the memory is valid until next call to NextRecord, NextChunk or NextBatch
boolean	NextChunk(InputSplit.Blob out_chunk) \brief get a chunk of memory that can contain multiple records, the caller needs to parse the content of the resulting chunk, for text file, out_chunk can contain data of multiple lines for recordio, out_chunk can contain multiple records(including headers) This function ensures there won't be partial record in the chunk caller can modify the memory content of out_chunk, the memory is valid until next call to NextRecord, NextChunk or NextBatch Usually NextRecord is sufficient, NextChunk can be used by some multi-threaded parsers to parse the input content
boolean	NextRecord(InputSplit.Blob out_rec) \brief get the next record, the returning value is valid until next call to NextRecord, NextChunk or NextBatch caller can modify the memory content of out_rec For text, out_rec contains a single line For recordio, out_rec contains one record content(with header striped)
void	ResetPartition(int part_index, int num_parts) \brief reset the Input split to a certain part id, The InputSplit will be pointed to the head of the new specified segment.

Methods inherited from class org.bytedeco.javacpp.Pointer

address, asBuffer, asByteBuffer, availablePhysicalBytes, calloc, capacity, capacity, close, deallocate, deallocate, deallocateReferences, deallocator, deallocator, equals, fill, formatBytes, free, getPointer, getPointer, getPointer, getPointer, hashCode, isNull, isNull, limit, limit, malloc, maxBytes, maxPhysicalBytes, memchr, memcmp, memcpy, memmove, memset, offsetAddress, offsetof, offsetof, parseBytes, physicalBytes, position, position, put, realloc, referenceCount, releaseReference, retainReference, setNull, sizeof, sizeof, toString, totalBytes, totalCount, totalPhysicalBytes, withDeallocator, zero

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

InputSplit

public InputSplit(Pointer p)

Pointer cast constructor. Invokes Pointer(Pointer).

Method Detail

HintChunkSize

public void HintChunkSize(@Cast(value="size_t")
                          long chunk_size)

\brief hint the inputsplit how large the chunk size it should return when implementing NextChunk this is a hint so may not be enforced, but InputSplit will try adjust its internal buffer size to the hinted value

Parameters:

chunk_size - the chunk size

GetTotalSize

@Cast(value="size_t")
public long GetTotalSize()

\brief get the total size of the InputSplit

BeforeFirst

public void BeforeFirst()

\brief reset the position of InputSplit to beginning

NextRecord

@Cast(value="bool")
public boolean NextRecord(InputSplit.Blob out_rec)

\brief get the next record, the returning value is valid until next call to NextRecord, NextChunk or NextBatch caller can modify the memory content of out_rec For text, out_rec contains a single line For recordio, out_rec contains one record content(with header striped)

Parameters:

out_rec - used to store the result

Returns:

true if we can successfully get next record false if we reached end of split

See Also:

for definition of record

NextChunk

@Cast(value="bool")
public boolean NextChunk(InputSplit.Blob out_chunk)

\brief get a chunk of memory that can contain multiple records, the caller needs to parse the content of the resulting chunk, for text file, out_chunk can contain data of multiple lines for recordio, out_chunk can contain multiple records(including headers) This function ensures there won't be partial record in the chunk caller can modify the memory content of out_chunk, the memory is valid until next call to NextRecord, NextChunk or NextBatch Usually NextRecord is sufficient, NextChunk can be used by some multi-threaded parsers to parse the input content

Parameters:

out_chunk - used to store the result

Returns:

true if we can successfully get next record false if we reached end of split

See Also:

for definition of record, to parse recordio content from out_chunk

NextBatch

@Cast(value="bool")
public boolean NextBatch(InputSplit.Blob out_chunk,
                                              @Cast(value="size_t")
                                              long n_records)

\brief get a chunk of memory that can contain multiple records, with hint for how many records is needed, the caller needs to parse the content of the resulting chunk, for text file, out_chunk can contain data of multiple lines for recordio, out_chunk can contain multiple records(including headers) This function ensures there won't be partial record in the chunk caller can modify the memory content of out_chunk, the memory is valid until next call to NextRecord, NextChunk or NextBatch

Parameters:

out_chunk - used to store the result

n_records - used as a hint for how many records should be returned, may be ignored

Returns:

true if we can successfully get next record false if we reached end of split

See Also:

for definition of record, to parse recordio content from out_chunk

ResetPartition

public void ResetPartition(@Cast(value="unsigned")
                           int part_index,
                           @Cast(value="unsigned")
                           int num_parts)

\brief reset the Input split to a certain part id, The InputSplit will be pointed to the head of the new specified segment. This feature may not be supported by every implementation of InputSplit.

Parameters:

part_index - The part id of the new input.

num_parts - The total number of parts.