ml.comet.experiment

Interface Experiment

All Superinterfaces:

AutoCloseable

All Known Subinterfaces:

ApiExperiment, OnlineExperiment

All Known Implementing Classes:

ApiExperimentImpl, OnlineExperimentImpl

public interface Experiment
extends AutoCloseable

The Experiment is a unit of measurable research that defines a single run with some data/parameters/code/results.

Creating an Experiment object in your code will report a new experiment to your Comet.ml project.

Your Experiment will automatically track and collect many things and will also allow you to manually report anything.

Method Summary

Modifier and Type	Method and Description
void	addTag(String tag) Adds a tag to this experiment.
void	end() Tells Comet that the Experiment is complete and release all associated resources.
List<LoggedExperimentAsset>	getAllAssetList() Get list of all logged experiment assets.
LoggedArtifact	getArtifact(String name) Allows looking for logged artifact using provided search parameters.
LoggedArtifact	getArtifact(String name, String workspace) Allows looking for logged artifact using provided search parameters.
LoggedArtifact	getArtifact(String name, String workspace, String versionOrAlias) Allows looking for logged artifact using provided search parameters.
List<LoggedExperimentAsset>	getAssetList(String type) Get list of the logged experiment assets with particular type.
String	getExperimentKey() Get the experiment key returned by Comet.
Optional<String>	getExperimentLink() Get the experiment link returned by Comet.
String	getExperimentName() Get experiment name.
GitMetaData	getGitMetadata() Get experiment git metadata.
Optional<String>	getGraph() Get experiment graph data.
Optional<String>	getHtml() Get experiment html.
List<Value>	getLogOther() Get experiment log other data.
ExperimentMetadata	getMetadata() Get experiment metadata.
List<Value>	getMetrics() Get experiment metrics.
Optional<String>	getOutput() Get experiment output data.
List<Value>	getParameters() Get experiment parameters.
String	getProjectName() Get project name of the experiment.
List<String>	getTags() Get experiment tags.
String	getWorkspaceName() Get workspace name of the experiment.
void	logCode(File file) Allows you to report code for the experiment.
void	logCode(File file, ExperimentContext context) Allows you to report code for the experiment.
void	logCode(String code, String logicalPath) Allows you to report code for the experiment.
void	logCode(String code, String logicalPath, ExperimentContext context) Allows you to report code for the experiment.
void	logCurve(Curve curve)
void	logCurve(Curve curve, boolean overwrite)
void	logCurve(Curve curve, boolean overwrite, ExperimentContext context) Allows to log x/y curve into your Comet experiment.
void	logEndTime(long endTimeMillis) Logs the start time of the experiment.
void	logGitMetadata(GitMetaData gitMetadata) Logs Git Metadata for the experiment.
void	logGraph(String graph) Logs a graph to Comet.
void	logHtml(String html, boolean override) Allows you to create html report for the experiment.
void	logLine(String line, long offset, boolean stderr, String context) Send logs to Comet.
void	logMetric(String metricName, Object metricValue, ExperimentContext context) Logs a metric with Comet.
void	logMetric(String metricName, Object metricValue, long step, long epoch) Logs a metric with Comet.
void	logOther(String key, Object value) Logs a key value pair with Comet.
void	logParameter(String parameterName, Object paramValue, ExperimentContext context) Logs a param with Comet.
void	logParameter(String parameterName, Object paramValue, long step) Logs a param with Comet.
void	logStartTime(long startTimeMillis) Logs the start time of the experiment.
void	logText(String text)
void	logText(String text, ExperimentContext context)
void	logText(String text, ExperimentContext context, Map<String,Object> metadata) Allows to log additional textual data associated with Comet Experiment.
void	setExperimentName(String experimentName) Sets the experiment name.
void	uploadAsset(File asset, boolean overwrite, ExperimentContext context) Upload an asset to be associated with the experiment, for example the trained weights of a neural net.
void	uploadAsset(File asset, boolean overwrite, long step, long epoch) Upload an asset to be associated with the experiment, for example the trained weights of a neural net.
void	uploadAsset(File asset, String logicalPath, boolean overwrite, ExperimentContext context) Upload an asset to be associated with the experiment, for example the trained weights of a neural net.
void	uploadAsset(File asset, String logicalPath, boolean overwrite, long step, long epoch) Upload an asset to be associated with the experiment, for example the trained weights of a neural net.

Methods inherited from interface java.lang.AutoCloseable

close

Method Detail

getExperimentKey

String getExperimentKey()

Get the experiment key returned by Comet.

Returns:

the experiment key assigned by Comet

getExperimentLink

Optional<String> getExperimentLink()

Get the experiment link returned by Comet.

Returns:

the experiment link assigned by Comet.

getProjectName

String getProjectName()

Get project name of the experiment.

Returns:

the project name of the experiment.

getWorkspaceName

String getWorkspaceName()

Get workspace name of the experiment.

Returns:

the workspace name of the experiment.

getExperimentName

String getExperimentName()

Get experiment name.

Returns:

the name of the experiment.

getMetadata

ExperimentMetadata getMetadata()

Get experiment metadata.

Returns:

the metadata associated with experiment.

getGitMetadata

GitMetaData getGitMetadata()

Get experiment git metadata.

Returns:

the GIT metadata for this experiment.

getHtml

Optional<String> getHtml()

Get experiment html.

Returns:

the HTML of the experiment.

getOutput

Optional<String> getOutput()

Get experiment output data.

Returns:

the output data of the experiment.

getGraph

Optional<String> getGraph()

Get experiment graph data.

Returns:

the graph data associated with experiment.

getParameters

List<Value> getParameters()

Get experiment parameters.

Returns:

the parameters logged by experiment.

getMetrics

List<Value> getMetrics()

Get experiment metrics.

Returns:

the metrics logged by experiment.

getLogOther

List<Value> getLogOther()

Get experiment log other data.

Returns:

the other data logged with experiment.

getTags

List<String> getTags()

Get experiment tags.

Returns:

the TAGs associated with experiment

getAssetList

List<LoggedExperimentAsset> getAssetList(String type)

Get list of the logged experiment assets with particular type.

Parameters:

type - the type of assets to be included.

Returns:

the list of assets associated with experiment.

getAllAssetList

List<LoggedExperimentAsset> getAllAssetList()

Get list of all logged experiment assets.

Returns:

the list of assets associated with experiment.

getArtifact

LoggedArtifact getArtifact(String name,
                           String workspace,
                           String versionOrAlias)
                    throws ArtifactException

Allows looking for logged artifact using provided search parameters.

Parameters:

name - the short name of the artifact like 'artifact-name'.

workspace - the workspace where artifact was saved.

versionOrAlias - the artifact version or alias to be returned.

Returns:

the LoggedArtifact instance.

Throws:

ArtifactException - if failed to get comet artifact.

getArtifact

LoggedArtifact getArtifact(String name,
                           String workspace)
                    throws ArtifactException

Allows looking for logged artifact using provided search parameters.

Parameters:

name - the short name of the artifact like 'artifact-name' or the name of artifact with version or alias like 'artifact-name:versionOrAlias'.

workspace - the workspace where artifact was saved.

Returns:

the LoggedArtifact instance.

Throws:

ArtifactException - if failed to get comet artifact.

getArtifact

LoggedArtifact getArtifact(String name)
                    throws ArtifactException

Allows looking for logged artifact using provided search parameters.

Parameters:

name - the name of the artifact. This could either be a fully qualified artifact name like 'workspace/artifact-name:versionOrAlias' or any short forms like 'artifact-name', 'artifact-name:versionOrAlias'.

Returns:

the LoggedArtifact instance.

Throws:

ArtifactException - if failed to get comet artifact.

setExperimentName

void setExperimentName(String experimentName)

Sets the experiment name.

Parameters:

experimentName - The new name for the experiment

logLine

void logLine(String line,
             long offset,
             boolean stderr,
             String context)

Send logs to Comet.

Parameters:

line - Text to be logged

offset - Offset describes the place for current text to be inserted

stderr - the flag to indicate if this is StdErr message.

context - the context to be associated with the parameter.

logMetric

void logMetric(String metricName,
               Object metricValue,
               ExperimentContext context)

Logs a metric with Comet. For running experiment updates current step to one from param! Metrics are generally values that change from step to step.

Parameters:

metricName - The name for the metric to be logged

metricValue - The new value for the metric. If the values for a metric are plottable we will plot them.

context - the context to be associated with the metric.

logMetric

void logMetric(String metricName,
               Object metricValue,
               long step,
               long epoch)

Logs a metric with Comet. For running experiment updates current step to one from param! Metrics are generally values that change from step to step.

Parameters:

metricName - The name for the metric to be logged

metricValue - The new value for the metric. If the values for a metric are plottable we will plot them

step - The current step for this metric, this will set the given step for this experiment

epoch - The current epoch for this metric, this will set the given epoch for this experiment.

logParameter

void logParameter(String parameterName,
                  Object paramValue,
                  ExperimentContext context)

Logs a param with Comet. For running experiment updates current step to one from param! Params should be set at the start of the experiment.

Parameters:

parameterName - The name of the param being logged

paramValue - The value for the param being logged

context - the context to be associated with the parameter.

logParameter

void logParameter(String parameterName,
                  Object paramValue,
                  long step)

Logs a param with Comet. For running experiment updates current step to one from param! Params should be set at the start of the experiment.

Parameters:

parameterName - The name of the param being logged

paramValue - The value for the param being logged

step - The current step for this metric, this will set the given step for this experiment.

logHtml

void logHtml(String html,
             boolean override)

Allows you to create html report for the experiment.

Parameters:

html - A block of html to be sent to Comet

override - Whether previous html sent should be deleted. If true the old html will be deleted. If false, it will have the new html appended onto the end.

logOther

void logOther(String key,
              Object value)

Logs a key value pair with Comet. This can be used for any sort of generic per experiment data you wish to track.

Parameters:

key - The key for the data to be stored

value - The value for said key

addTag

void addTag(String tag)

Adds a tag to this experiment.

Parameters:

tag - The tag to be added

logGraph

void logGraph(String graph)

Logs a graph to Comet.

Parameters:

graph - The graph to be logged

logStartTime

void logStartTime(long startTimeMillis)

Logs the start time of the experiment.

Parameters:

startTimeMillis - When you want to say that the experiment started

logEndTime

void logEndTime(long endTimeMillis)

Logs the start time of the experiment.

Parameters:

endTimeMillis - When you want to say that the experiment ended

logCode

void logCode(String code,
             String logicalPath,
             ExperimentContext context)

Allows you to report code for the experiment.

Parameters:

code - Code to be sent to Comet

logicalPath - Name of source file to be displayed on UI 'code' tab

context - the context to be associated with the asset.

logCode

void logCode(String code,
             String logicalPath)

Allows you to report code for the experiment.

Parameters:

code - Code to be sent to Comet

logicalPath - Name of source file to be displayed on UI 'code' tab

logCode

void logCode(File file,
             ExperimentContext context)

Allows you to report code for the experiment.

Parameters:

file - Asset with source code to be sent

context - the context to be associated with the asset.

logCode

void logCode(File file)

Allows you to report code for the experiment.

Parameters:

file - Asset with source code to be sent

logText

void logText(String text,
             ExperimentContext context,
             Map<String,Object> metadata)

Allows to log additional textual data associated with Comet Experiment. These strings appear on the Text Tab in the Comet UI.

Parameters:

text - the text data to be logged.

context - the current experiment context to be associated.

metadata - the additional metadata for the text.

logText

void logText(String text,
             ExperimentContext context)

logText

void logText(String text)

logCurve

void logCurve(Curve curve,
              boolean overwrite,
              ExperimentContext context)

Allows to log x/y curve into your Comet experiment. This can be, for example, the time-series data, etc.

Parameters:

curve - the Curve object holding the data points.

overwrite - allows to override the previously logged curve with the same name.

context - the experiment context to be associated with data record (step, epoch, context ID).

logCurve

void logCurve(Curve curve,
              boolean overwrite)

logCurve

void logCurve(Curve curve)

uploadAsset

void uploadAsset(File asset,
                 String logicalPath,
                 boolean overwrite,
                 ExperimentContext context)

Upload an asset to be associated with the experiment, for example the trained weights of a neural net. For running experiment updates current step to one from param!

Parameters:

asset - The asset to be stored

logicalPath - The file name under which the asset should be stored in Comet. E.g. "someFile.txt"

overwrite - Whether to overwrite files of the same name in Comet

context - the context to be associated with the asset.

uploadAsset

void uploadAsset(File asset,
                 String logicalPath,
                 boolean overwrite,
                 long step,
                 long epoch)

Upload an asset to be associated with the experiment, for example the trained weights of a neural net. For running experiment updates current step to one from param!

Parameters:

asset - The asset to be stored

logicalPath - The file name under which the asset should be stored in Comet. E.g. "someFile.txt"

overwrite - Whether to overwrite files of the same name in Comet

step - the step to be associated with the asset

epoch - the epoch to be associated with the asset

uploadAsset

void uploadAsset(File asset,
                 boolean overwrite,
                 ExperimentContext context)

Upload an asset to be associated with the experiment, for example the trained weights of a neural net. For running experiment updates current step to one from param!

Parameters:

asset - The file asset to be stored. The name of the file will be used as assets identifier on Comet.

overwrite - Whether to overwrite files of the same name in Comet

context - the context to be associated with the asset.

uploadAsset

void uploadAsset(File asset,
                 boolean overwrite,
                 long step,
                 long epoch)

Upload an asset to be associated with the experiment, for example the trained weights of a neural net. For running experiment updates current step to one from param!

Parameters:

asset - The file asset to be stored. The name of the file will be used as assets identifier on Comet.

overwrite - Whether to overwrite files of the same name in Comet

step - the step to be associated with the asset

epoch - the epoch to be associated with the asset

logGitMetadata

void logGitMetadata(GitMetaData gitMetadata)

Logs Git Metadata for the experiment.

Parameters:

gitMetadata - The Git Metadata for the experiment

end

void end()

Tells Comet that the Experiment is complete and release all associated resources.