edu.upc.dama.dex.io
Class Loader

java.lang.Object
  edu.upc.dama.dex.io.Loader

Direct Known Subclasses:

EdgeTypeLoader, NodeTypeLoader

public abstract class Loader
extends java.lang.Object

Loads data in a node or edge type.

It allows for store data structured as rows in a node or edge type of a Graph.

Since data is structured as rows (CSV files, etc), it uses a RowReader to get all the data.

When the load is running a Loader.LoaderEvent is sent to their listeners with a given frequency.

Author:

Sparsity Technologies

Nested Class Summary

class	Loader.LoaderEvent Constains information of the running Loader.
static class	Loader.Mode Load can work in different ways.

Field Summary

static java.lang.String	DEFAULT_DATEFORMAT Default date format.
protected long	headAttr Attribute identifier in the Graph to find head nodes when loading data for an edge type.
protected int	headPos Column index in the RowReader which stores the heads when loading data for an edge type.
protected int	headType Type of the attribute head (edge only)
protected long	tailAttr Attribute identifier in the Graph to find tail nodes when loading data for an edge type.
protected int	tailPos Column index in the RowReader which stores the tails when loading data for an edge type.
protected int	tailType Type of the attribute tail (edge only)

Constructor Summary

protected

Loader(RowReader rowReader, Graph graph, int objectType) Creates a new instance of Loader.

Method Summary

int	getFrequency() Gets the frequency when Loader.LoaderEvents are sent to their LoaderListener.
java.io.File	getLogFile() Gets the File used to save the errors while loading.
void	registerLoaderListener(LoaderListener ll) Adds a new LoaderListener.
void	run(Loader.Mode phases, int partitions) Starts the load process.
void	setAttribute(long attributeId, int column) Sets information to load a column, that is an attribute.
void	setDateFormat(java.lang.String dateformat) Sets the date format to parse Value.TIMESTAMP values.
void	setDeleteTemporalFiles(boolean delete) Sets the delete temporary files policy.
void	setFrequency(int frequency) Updates the frequency of the Loader.LoaderEvents.
void	setLogFile(java.io.File file) Sets the file used to save the errors while loading.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_DATEFORMAT

public static final java.lang.String DEFAULT_DATEFORMAT

Default date format.

By default, date format parses values with the following format: yyyy-MM-dd hh:mm:ss.

See Also:

Constant Field Values

tailPos

protected int tailPos

Column index in the RowReader which stores the tails when loading data for an edge type.

headPos

protected int headPos

Column index in the RowReader which stores the heads when loading data for an edge type.

headAttr

protected long headAttr

Attribute identifier in the Graph to find head nodes when loading data for an edge type.

tailAttr

protected long tailAttr

Attribute identifier in the Graph to find tail nodes when loading data for an edge type.

tailType

protected int tailType

Type of the attribute tail (edge only)

headType

protected int headType

Type of the attribute head (edge only)

Constructor Detail

Loader

protected Loader(RowReader rowReader,
                 Graph graph,
                 int objectType)

Creates a new instance of Loader.

Parameters:

rowReader - RowReader to get the data from.

graph - Graph to load the data to.

objectType - Node or edge type to load the data to.

Method Detail

registerLoaderListener

public void registerLoaderListener(LoaderListener ll)

Adds a new LoaderListener.

Parameters:

ll - A LoaderListener to be registered.

setFrequency

public void setFrequency(int frequency)

Updates the frequency of the Loader.LoaderEvents.

Parameters:

frequency - New frequency value.

getFrequency

public int getFrequency()

Gets the frequency when Loader.LoaderEvents are sent to their LoaderListener.

Returns:

The frequency.

setDateFormat

public void setDateFormat(java.lang.String dateformat)

Sets the date format to parse Value.TIMESTAMP values.

Parameters:

dateformat - String with the new date format.

See Also:

SimpleDateFormat

setAttribute

public void setAttribute(long attributeId,
                         int column)

Sets information to load a column, that is an attribute.

If the RowReader has N columns, then 0 <= column < N.

Parameters:

attributeId - Attribute identifier in the Graph.

column - Column index in the RowReader where values for the attributeId are stored.

setDeleteTemporalFiles

public void setDeleteTemporalFiles(boolean delete)

Sets the delete temporary files policy.

Parameters:

delete - If true, all temporary files are deleted when the execution finishes, otherwise they will not be deleted.

setLogFile

public void setLogFile(java.io.File file)
                throws java.io.IOException

Sets the file used to save the errors while loading.

Parameters:

file - File used to write the errors. If file == null, errors creating or setting objects will generate Exceptions. If not, errors generate a report to the given file.

Throws:

java.io.IOException - If bad things happen with the given file.

getLogFile

public java.io.File getLogFile()

Gets the File used to save the errors while loading.

Returns:

Log file with the errors generated during the load.

run

public void run(Loader.Mode phases,
                int partitions)
         throws java.io.IOException

Starts the load process.

Only when working in mode Loader.Mode.N_PHASES, load can be partitioned horizontally.

Parameters:

phases - Number of phases to perform the load.

partitions - Number of horizontal partitions to perform the load.

Throws:

java.io.IOException

See Also:

Loader.Mode