ml.shifu.guagua.example.nn
类 MemoryDiskMLDataSet

java.lang.Object
  ml.shifu.guagua.example.nn.MemoryDiskMLDataSet

所有已实现的接口：

Iterable<org.encog.ml.data.MLDataPair>, org.encog.ml.data.MLDataSet

public class MemoryDiskMLDataSet
extends Object
implements org.encog.ml.data.MLDataSet

A hybrid data set combining BasicMLDataSet and BufferedMLDataSet together.

With this data set, element is added firstly in memory, if over maxByteSize then element will be added into disk.

This data set provide a very important feature to make in memory computing more stable. Even for some cases no enough memory, memory and disk will be leveraged together to accelerate computing.

Example almost same as BufferedMLDataSet:

 MemoryDiskMLDataSet dataSet = new MemoryDiskMLDataSet(400, "a.txt");
 dataSet.beginLoad(10, 1);
 dataSet.add(pair);
 dataSet.endLoad();
 
 Iterator iterator = dataSet.iterator();
 while(iterator.hasNext()) {
     MLDataPair next = iterator.next();
     ...
 }
 
 dataSet.close();
 

作者：

Zhang David (pengzhang@paypal.com)

构造方法摘要

MemoryDiskMLDataSet(long maxByteSize, String fileName)
Constructor with maxByteSize and fileName

MemoryDiskMLDataSet(long maxByteSize, String fileName, int inputCount, int outputCount)
Constructor with maxByteSize, fileName, inputCount and outputCount.

MemoryDiskMLDataSet(String fileName, int inputCount, int outputCount)
Constructor with fileName, inputCount and outputCount

方法摘要

void	add(org.encog.ml.data.MLData data)
void	add(org.encog.ml.data.MLData inputData, org.encog.ml.data.MLData idealData)
void	add(org.encog.ml.data.MLDataPair inputData)
void	beginLoad(int inputSize, int idealSize) Setting input variable size and output target size.
void	close()
void	endLoad() This method should be called once all the data has been loaded.
long	getDiskCount()
int	getIdealSize()
int	getInputSize()
long	getMemoryCount()
void	getRecord(long index, org.encog.ml.data.MLDataPair pair)
long	getRecordCount()
boolean	isSupervised()
Iterator<org.encog.ml.data.MLDataPair>	iterator()
static void	main(String[] args)
org.encog.ml.data.MLDataSet	openAdditional()

从类 java.lang.Object 继承的方法

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造方法详细信息

MemoryDiskMLDataSet

public MemoryDiskMLDataSet(String fileName,
                           int inputCount,
                           int outputCount)

Constructor with fileName, inputCount and outputCount

MemoryDiskMLDataSet

public MemoryDiskMLDataSet(long maxByteSize,
                           String fileName)

Constructor with maxByteSize and fileName

MemoryDiskMLDataSet

public MemoryDiskMLDataSet(long maxByteSize,
                           String fileName,
                           int inputCount,
                           int outputCount)

Constructor with maxByteSize, fileName, inputCount and outputCount.

方法详细信息

beginLoad

public final void beginLoad(int inputSize,
                            int idealSize)

Setting input variable size and output target size.

参数：

inputSize - input variable size

idealSize - output target size

endLoad

public final void endLoad()

This method should be called once all the data has been loaded. The underlying file will be closed. The binary fill will then be opened for reading.

iterator

public Iterator<org.encog.ml.data.MLDataPair> iterator()

指定者：

接口 Iterable<org.encog.ml.data.MLDataPair> 中的 iterator

getIdealSize

public int getIdealSize()

指定者：

接口 org.encog.ml.data.MLDataSet 中的 getIdealSize

getInputSize

public int getInputSize()

指定者：

接口 org.encog.ml.data.MLDataSet 中的 getInputSize

isSupervised

public boolean isSupervised()

指定者：

接口 org.encog.ml.data.MLDataSet 中的 isSupervised

getRecordCount

public long getRecordCount()

指定者：

接口 org.encog.ml.data.MLDataSet 中的 getRecordCount

getRecord

public void getRecord(long index,
                      org.encog.ml.data.MLDataPair pair)

指定者：

接口 org.encog.ml.data.MLDataSet 中的 getRecord

openAdditional

public org.encog.ml.data.MLDataSet openAdditional()

指定者：

接口 org.encog.ml.data.MLDataSet 中的 openAdditional

add

public void add(org.encog.ml.data.MLData data)

指定者：

接口 org.encog.ml.data.MLDataSet 中的 add

add

public void add(org.encog.ml.data.MLData inputData,
                org.encog.ml.data.MLData idealData)

指定者：

接口 org.encog.ml.data.MLDataSet 中的 add

add

public void add(org.encog.ml.data.MLDataPair inputData)

指定者：

接口 org.encog.ml.data.MLDataSet 中的 add

close

public void close()

指定者：

接口 org.encog.ml.data.MLDataSet 中的 close

getMemoryCount

public long getMemoryCount()

返回：

the memoryCount

getDiskCount

public long getDiskCount()

返回：

the diskCount

main

public static void main(String[] args)