de.gsi.dataset.serializer.spi.iobuffer

Class DataSetSerialiser

java.lang.Object

de.gsi.dataset.utils.DataSetUtilsHelper

de.gsi.dataset.serializer.spi.iobuffer.DataSetSerialiser

public class DataSetSerialiser
extends DataSetUtilsHelper

Class to efficiently serialise and de-serialise DataSet objects into binary byte arrays. The performance can be tuned through:

• using floats (ie. memory-IO vs network-IO bound serialisation), or
• via setDataLablesSerialised(boolean) (default: false) to control whether data labels and styles shall be processed
• via setMetaDataSerialised(boolean) (default: true) to control whether meta data shall be processed

Author:

rstein

Field Summary

Fields inherited from class de.gsi.dataset.utils.DataSetUtilsHelper

BYTE_ARRAY_CACHE_LOCK, byteArrayCache, STRING_BUFFER_CACHE_LOCK, stringBuilderCache

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	DataSetSerialiser()

Method Summary

Modifier and Type	Method and Description
protected static Optional<FieldHeader>	checkFieldCompatibility(IoBuffer buffer, List<FieldHeader> fieldHeaderList, String fieldName, DataType... requireDataTypes)
static boolean	isDataLablesSerialised()
static boolean	isMetaDataSerialised()
protected static void	parseDataLabels(IoBuffer readBuffer, DataSetBuilder builder, List<FieldHeader> fieldHeaderList)
protected static void	parseHeader(IoBuffer readBuffer, DataSetBuilder builder, List<FieldHeader> fieldHeaderList)
protected static void	parseMetaData(IoBuffer readBuffer, DataSetBuilder builder, List<FieldHeader> fieldHeaderList)
protected static void	parseNumericData(IoBuffer readBuffer, DataSetBuilder builder, List<FieldHeader> fieldHeaderList)
static DataSet	readDataSetFromByteArray(IoBuffer readBuffer) Read a Dataset from a byte array containing comma separated values. The data format is a custom extension of csv with an additional #-commented Metadata Header and a $-commented column header.
static void	setDataLablesSerialised(boolean state)
static void	setMetaDataSerialised(boolean state)
protected static void	writeDataLabelsToStream(IoBuffer buffer, DataSet dataSet)
static void	writeDataSetToByteArray(DataSet dataSet, IoBuffer buffer, boolean asFloat) Write data set into byte buffer.
protected static void	writeHeaderDataToStream(IoBuffer buffer, DataSet dataSet)
protected static void	writeMetaDataToStream(IoBuffer buffer, DataSet dataSet)
protected static void	writeNumericBinaryDataToBuffer(IoBuffer buffer, DataSet dataSet, boolean asFloat)

Methods inherited from class de.gsi.dataset.utils.DataSetUtilsHelper

getCachedDoubleArray, getCachedStringBuilder, integralSimple, mean, readDoubleArrayFromBuffer, release, release, rootMeanSquare, toDoubleArray, toFloatArray, writeDoubleArrayAsFloatToByteBuffer, writeDoubleArrayToByteBuffer

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DataSetSerialiser

protected DataSetSerialiser()

Method Detail

checkFieldCompatibility

protected static Optional<FieldHeader> checkFieldCompatibility(IoBuffer buffer,
                                                               List<FieldHeader> fieldHeaderList,
                                                               String fieldName,
                                                               DataType... requireDataTypes)

isDataLablesSerialised

public static boolean isDataLablesSerialised()

isMetaDataSerialised

public static boolean isMetaDataSerialised()

parseDataLabels

protected static void parseDataLabels(IoBuffer readBuffer,
                                      DataSetBuilder builder,
                                      List<FieldHeader> fieldHeaderList)

parseHeader

protected static void parseHeader(IoBuffer readBuffer,
                                  DataSetBuilder builder,
                                  List<FieldHeader> fieldHeaderList)

parseMetaData

protected static void parseMetaData(IoBuffer readBuffer,
                                    DataSetBuilder builder,
                                    List<FieldHeader> fieldHeaderList)

parseNumericData

protected static void parseNumericData(IoBuffer readBuffer,
                                       DataSetBuilder builder,
                                       List<FieldHeader> fieldHeaderList)

readDataSetFromByteArray

public static DataSet readDataSetFromByteArray(IoBuffer readBuffer)

Read a Dataset from a byte array containing comma separated values.
The data format is a custom extension of csv with an additional #-commented Metadata Header and a $-commented column header. Expects the following columns in this order to be present: index, x, y, eyn, eyp.

Parameters:

readBuffer - IoBuffer (encapsulates byte array).

Returns:

DataSet with the data and metadata read from the file

setDataLablesSerialised

public static void setDataLablesSerialised(boolean state)

setMetaDataSerialised

public static void setMetaDataSerialised(boolean state)

writeDataLabelsToStream

protected static void writeDataLabelsToStream(IoBuffer buffer,
                                              DataSet dataSet)

writeDataSetToByteArray

public static void writeDataSetToByteArray(DataSet dataSet,
                                           IoBuffer buffer,
                                           boolean asFloat)

Write data set into byte buffer.

Parameters:

dataSet - The DataSet to export

buffer - byte output buffer (N.B. keep caching this object)

asFloat - true: encode data as binary floats (smaller size, performance), or false as double (better precision)

writeHeaderDataToStream

protected static void writeHeaderDataToStream(IoBuffer buffer,
                                              DataSet dataSet)

writeMetaDataToStream

protected static void writeMetaDataToStream(IoBuffer buffer,
                                            DataSet dataSet)

writeNumericBinaryDataToBuffer

protected static void writeNumericBinaryDataToBuffer(IoBuffer buffer,
                                                     DataSet dataSet,
                                                     boolean asFloat)

Parameters:

buffer - IoBuffer to write binary data into

dataSet - to be exported

asFloat - true use 32-bit floats (less memory, faster transfer) instead of 64-bit doubles (DataSet default, higher precision)