ml.comet.experiment

Interface OnlineExperiment

All Superinterfaces:

AutoCloseable, Experiment

All Known Implementing Classes:

OnlineExperimentImpl

public interface OnlineExperiment
extends Experiment

The OnlineExperiment should be used to asynchronously update data of your Comet.ml experiment.

This experiment type allows you to automatically intercept StdOut and StdErr streams and send them to the Comet.ml. Use the setInterceptStdout() to start automatic interception of StdOut and the stopInterceptStdout() to stop.

Also, it is possible to use setStep(long), setEpoch(long), and setContext(String) which will bbe automatically associated with related logged data records.

Method Summary

Modifier and Type	Method and Description
String	getContext() Gets the current context identifier as recorded in the OnlineExperiment object locally.
long	getEpoch() Gets the current epoch as recorded by the Experiment object locally.
long	getStep() Gets the current step as recorded by the Experiment object locally.
CompletableFuture<LoggedArtifact>	logArtifact(Artifact artifact) Logs an Artifact object.
void	logAssetFolder(File folder, boolean logFilePath)
void	logAssetFolder(File folder, boolean logFilePath, boolean recursive)
void	logAssetFolder(File folder, boolean logFilePath, boolean recursive, ExperimentContext context) Logs all the files located in the given folder as assets.
void	logLine(String line, long offset, boolean stderr) Send output logs to Comet.
void	logMetric(String metricName, Object metricValue)
void	logMetric(String metricName, Object metricValue, long step) Logs a metric with Comet.
void	logModel(String modelName, byte[] data, String logicalPath)
void	logModel(String modelName, byte[] data, String logicalPath, boolean overwrite)
void	logModel(String modelName, byte[] data, String logicalPath, boolean overwrite, Map<String,Object> metadata)
void	logModel(String modelName, byte[] data, String logicalPath, boolean overwrite, Map<String,Object> metadata, ExperimentContext context) Logs the model data from specified file.
void	logModel(String modelName, File file)
void	logModel(String modelName, File file, String logicalPath)
void	logModel(String modelName, File file, String logicalPath, boolean overwrite)
void	logModel(String modelName, File file, String logicalPath, boolean overwrite, Map<String,Object> metadata)
void	logModel(String modelName, File file, String logicalPath, boolean overwrite, Map<String,Object> metadata, ExperimentContext context) Logs the model data from specified file.
void	logModelFolder(String modelName, File folder)
void	logModelFolder(String modelName, File folder, boolean logFilePath, Map<String,Object> metadata)
void	logModelFolder(String modelName, File folder, boolean logFilePath, Map<String,Object> metadata, ExperimentContext context) Logs the model data from specified folder.
void	logModelFolder(String modelName, File folder, Map<String,Object> metadata)
void	logParameter(String parameterName, Object paramValue) Logs a param with Comet under the current experiment step.
void	logRemoteAsset(URI uri, boolean overwrite)
void	logRemoteAsset(URI uri, String logicalPath, boolean overwrite)
void	logRemoteAsset(URI uri, String logicalPath, boolean overwrite, Map<String,Object> metadata)
void	logRemoteAsset(URI uri, String logicalPath, boolean overwrite, Map<String,Object> metadata, ExperimentContext context) Logs a Remote Asset identified by a URI.
void	nextEpoch() Increments the epoch that the experiment is on.
void	nextStep() Increments the step that the experiment is on.
void	setContext(String context) Sets the context identifier for any logs and uploaded files.
void	setEpoch(long epoch) Sets the current epoch for the experiment.
void	setInterceptStdout() Turn on intercept of stdout and stderr and the logging of both in Comet.
void	setStep(long step) Sets the current step for the experiment.
void	stopInterceptStdout() Turn off intercept of stdout and stderr and turns off their logging in Comet.
void	uploadAsset(File asset, boolean overwrite)
void	uploadAsset(File asset, String logicalPath, boolean overwrite)
void	uploadAsset(File asset, String logicalPath, boolean overwrite, long step) Upload an asset under the current experiment step to be associated with the experiment, for example the trained weights of a neural net.

Methods inherited from interface ml.comet.experiment.Experiment

addTag, end, getAllAssetList, getArtifact, getArtifact, getArtifact, getAssetList, getExperimentKey, getExperimentLink, getExperimentName, getGitMetadata, getGraph, getHtml, getLogOther, getMetadata, getMetrics, getOutput, getParameters, getProjectName, getTags, getWorkspaceName, logCode, logCode, logCode, logCode, logEndTime, logGitMetadata, logGraph, logHtml, logLine, logMetric, logMetric, logOther, logParameter, logParameter, logStartTime, setExperimentName, uploadAsset, uploadAsset, uploadAsset, uploadAsset

Methods inherited from interface java.lang.AutoCloseable

close

Method Detail

setInterceptStdout

void setInterceptStdout()
                 throws IOException

Turn on intercept of stdout and stderr and the logging of both in Comet.

Throws:

IOException - if an I/O exception occurs.

stopInterceptStdout

void stopInterceptStdout()
                  throws IOException

Turn off intercept of stdout and stderr and turns off their logging in Comet.

Throws:

IOException - if an I/O exception occurs.

setStep

void setStep(long step)

Sets the current step for the experiment. This is used when logging metrics and params.

Parameters:

step - the current step of the experiment.

nextStep

void nextStep()

Increments the step that the experiment is on.

getStep

long getStep()

Gets the current step as recorded by the Experiment object locally.

Returns:

the current step of the experiment

setEpoch

void setEpoch(long epoch)

Sets the current epoch for the experiment.

Parameters:

epoch - the current epoch for the experiment.

nextEpoch

void nextEpoch()

Increments the epoch that the experiment is on.

getEpoch

long getEpoch()

Gets the current epoch as recorded by the Experiment object locally.

Returns:

the current epoch of the experiment.

setContext

void setContext(String context)

Sets the context identifier for any logs and uploaded files.

Parameters:

context - the context identifier to be associated with any log records, files, and assets.

getContext

String getContext()

Gets the current context identifier as recorded in the OnlineExperiment object locally.

Returns:

the current context which associated with log records of this experiment.

logMetric

void logMetric(String metricName,
               Object metricValue,
               long step)

Logs a metric with Comet. For running experiment updates current step to one from param! Metrics are generally values that change from step to step.

Parameters:

metricName - The name for the metric to be logged

metricValue - The new value for the metric. If the values for a metric are plottable we will plot them

step - The current step for this metric, this will set the given step for this experiment

logMetric

void logMetric(String metricName,
               Object metricValue)

logParameter

void logParameter(String parameterName,
                  Object paramValue)

Logs a param with Comet under the current experiment step. Params should be set at the start of the experiment.

Parameters:

parameterName - The name of the param being logged

paramValue - The value for the param being logged

logLine

void logLine(String line,
             long offset,
             boolean stderr)

Send output logs to Comet.

Parameters:

line - Text to be logged

offset - Offset describes the place for current text to be inserted

stderr - the flag to indicate if this is StdErr message.

uploadAsset

void uploadAsset(File asset,
                 String logicalPath,
                 boolean overwrite,
                 long step)

Upload an asset under the current experiment step to be associated with the experiment, for example the trained weights of a neural net.

Parameters:

asset - The asset to be stored

logicalPath - The file name under which the asset should be stored in Comet. E.g. "someFile.txt"

overwrite - Whether to overwrite files of the same name in Comet

step - The step to be associated with asset

uploadAsset

void uploadAsset(File asset,
                 String logicalPath,
                 boolean overwrite)

uploadAsset

void uploadAsset(File asset,
                 boolean overwrite)

logAssetFolder

void logAssetFolder(File folder,
                    boolean logFilePath,
                    boolean recursive,
                    ExperimentContext context)

Logs all the files located in the given folder as assets.

Parameters:

folder - the folder you want to log.

logFilePath - if true, log the file path with each file.

recursive - if true, recurse the folder.

context - the experiment context to be associated with the logged assets.

logAssetFolder

void logAssetFolder(File folder,
                    boolean logFilePath,
                    boolean recursive)

logAssetFolder

void logAssetFolder(File folder,
                    boolean logFilePath)

logRemoteAsset

void logRemoteAsset(URI uri,
                    String logicalPath,
                    boolean overwrite,
                    Map<String,Object> metadata,
                    ExperimentContext context)

Logs a Remote Asset identified by a URI. A Remote Asset is an asset but its content is not uploaded and stored on Comet. Rather a link for its location is stored, so you can identify and distinguish between two experiment using different version of a dataset stored somewhere else.

Parameters:

uri - the URI pointing to the remote asset location. There is no imposed format, and it could be a private link.

logicalPath - the optional "name" of the remote asset, could be a dataset name, a model file name.

overwrite - if true will overwrite all existing assets with the same name.

metadata - Some additional data to attach to the remote asset. The dictionary values must be JSON compatible.

context - the experiment context to be associated with the logged assets.

logRemoteAsset

void logRemoteAsset(URI uri,
                    String logicalPath,
                    boolean overwrite,
                    Map<String,Object> metadata)

logRemoteAsset

void logRemoteAsset(URI uri,
                    String logicalPath,
                    boolean overwrite)

logRemoteAsset

void logRemoteAsset(URI uri,
                    boolean overwrite)

logArtifact

CompletableFuture<LoggedArtifact> logArtifact(Artifact artifact)
                                       throws ArtifactException

Logs an Artifact object. First, it creates a new version of the artifact. After that, it uploads asynchronously all the local and remote assets attached to the artifact object.

Parameters:

artifact - the Artifact instance.

Returns:

CompletableFuture which can be used to query for LoggedArtifact instance with details about new version of artifact that was logged.

Throws:

ArtifactException - if operation failed.

logModelFolder

void logModelFolder(String modelName,
                    File folder,
                    boolean logFilePath,
                    Map<String,Object> metadata,
                    ExperimentContext context)

Logs the model data from specified folder.

Parameters:

modelName - the name of the model.

folder - the folder with model's data files.

logFilePath - if true than prefix file path of each data file with folder name.

metadata - the metadata to be associated with asset files of the model. The dictionary values must be JSON compatible.

context - the experiment context to be associated with the logged model.

logModelFolder

void logModelFolder(String modelName,
                    File folder,
                    boolean logFilePath,
                    Map<String,Object> metadata)

logModelFolder

void logModelFolder(String modelName,
                    File folder,
                    Map<String,Object> metadata)

logModelFolder

void logModelFolder(String modelName,
                    File folder)

logModel

void logModel(String modelName,
              File file,
              String logicalPath,
              boolean overwrite,
              Map<String,Object> metadata,
              ExperimentContext context)

Logs the model data from specified file.

Parameters:

modelName - the name of the model

file - the file with model data.

logicalPath - the optional "name" of the model file, if missed the file name will be used.

overwrite - if true will overwrite all existing model with the same name.

metadata - the additional metadata to attach to the asset file of the model. The dictionary values must be JSON compatible.

context - the experiment context to be associated with the logged model.

logModel

void logModel(String modelName,
              File file,
              String logicalPath,
              boolean overwrite,
              Map<String,Object> metadata)

logModel

void logModel(String modelName,
              File file,
              String logicalPath,
              boolean overwrite)

logModel

void logModel(String modelName,
              File file,
              String logicalPath)

logModel

void logModel(String modelName,
              File file)

logModel

void logModel(String modelName,
              byte[] data,
              String logicalPath,
              boolean overwrite,
              Map<String,Object> metadata,
              ExperimentContext context)

Logs the model data from specified file.

Parameters:

modelName - the name of the model

data - the model's data.

logicalPath - the name of the model file.

overwrite - if true will overwrite all existing model with the same name.

metadata - the additional metadata to attach to the asset data of the model. The dictionary values must be JSON compatible.

context - the experiment context to be associated with the logged model.

logModel

void logModel(String modelName,
              byte[] data,
              String logicalPath,
              boolean overwrite,
              Map<String,Object> metadata)

logModel

void logModel(String modelName,
              byte[] data,
              String logicalPath,
              boolean overwrite)

logModel

void logModel(String modelName,
              byte[] data,
              String logicalPath)