ml.comet.experiment

Class BaseExperiment

java.lang.Object

ml.comet.experiment.BaseExperiment

All Implemented Interfaces:

Experiment

Direct Known Subclasses:

ApiExperiment, OnlineExperimentImpl

public abstract class BaseExperiment
extends Object
implements Experiment

The base class for all experiment implementations providing implementation of common routines.

Constructor Summary

Constructors

Constructor and Description
BaseExperiment()

Method Summary

Modifier and Type	Method and Description
void	addTag(@NonNull String tag) Adds a tag to this experiment.
List<ExperimentAssetLink>	getAssetList(@NonNull AssetType type) Get experiment asset list.
protected abstract Connection	getConnection() Returns connection associated with particular experiment.
protected abstract String	getContext() Gets the current context as recorded in the Experiment object locally.
GitMetadataRest	getGitMetadata() Get experiment git metadata.
Optional<String>	getGraph() Get experiment graph data.
Optional<String>	getHtml() Get experiment html.
protected abstract org.slf4j.Logger	getLogger() Returns logger instance associated with particular experiment.
List<ValueMinMaxDto>	getLogOther() Get experiment log other data.
ExperimentMetadataRest	getMetadata() Get experiment metadata.
List<ValueMinMaxDto>	getMetrics() Get experiment metrics.
Optional<String>	getOutput() Get experiment output data.
List<ValueMinMaxDto>	getParameters() Get experiment parameters.
List<String>	getTags() Get experiment tags.
void	logCode(@NonNull File asset) Allows you to report code for the experiment.
void	logCode(@NonNull String code, @NonNull String fileName) Allows you to report code for the experiment.
void	logEndTime(long endTimeMillis) Logs the start time of the experiment.
void	logGitMetadata(CreateGitMetadata gitMetadata) Logs Git Metadata for the experiment.
void	logGraph(@NonNull String graph) Logs a graph to Comet.
void	logHtml(@NonNull String html, boolean override) Allows you to create html report for the experiment.
void	logMetric(@NonNull String metricName, @NonNull Object metricValue, long step, long epoch) Logs a metric with Comet.
void	logOther(@NonNull String key, @NonNull Object value) Logs a key value pair with Comet.
void	logParameter(@NonNull String parameterName, @NonNull Object paramValue, long step) Logs a param with Comet.
void	logStartTime(long startTimeMillis) Logs the start time of the experiment.
void	setExperimentName(@NonNull String experimentName) Sets the experiment name.
void	uploadAsset(@NonNull File asset, boolean overwrite, long step, long epoch)
void	uploadAsset(@NonNull File asset, @NonNull String fileName, boolean overwrite, long step, long epoch) Upload an asset to be associated with the experiment, for example the trained weights of a neural net.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface ml.comet.experiment.Experiment

end, getExperimentKey, getExperimentLink, getExperimentName, getProjectName, getWorkspaceName

Constructor Detail

BaseExperiment

public BaseExperiment()

Method Detail

getContext

protected abstract String getContext()

Gets the current context as recorded in the Experiment object locally.

Returns:

the current context which associated with log records of this experiment. TODO: 03.11.2021 this can be made Optional

getConnection

protected abstract Connection getConnection()

Returns connection associated with particular experiment. The subclasses must override this method to provide relevant connection instance.

Returns:

the initialized connection associated with particular experiment.

getLogger

protected abstract org.slf4j.Logger getLogger()

Returns logger instance associated with particular experiment. The subclasses should override this method to provide specific logger instance.

Returns:

the logger instance associated with particular experiment.

setExperimentName

public void setExperimentName(@NonNull
                              @NonNull String experimentName)

Description copied from interface: Experiment

Sets the experiment name.

Specified by:

setExperimentName in interface Experiment

Parameters:

experimentName - The new name for the experiment

logMetric

public void logMetric(@NonNull
                      @NonNull String metricName,
                      @NonNull
                      @NonNull Object metricValue,
                      long step,
                      long epoch)

Description copied from interface: Experiment

Logs a metric with Comet. For running experiment updates current step to one from param! Metrics are generally values that change from step to step.

Specified by:

logMetric in interface Experiment

Parameters:

metricName - The name for the metric to be logged

metricValue - The new value for the metric. If the values for a metric are plottable we will plot them

step - The current step for this metric, this will set the given step for this experiment

epoch - The current epoch for this metric, this will set the given epoch for this experiment

logParameter

public void logParameter(@NonNull
                         @NonNull String parameterName,
                         @NonNull
                         @NonNull Object paramValue,
                         long step)

Description copied from interface: Experiment

Logs a param with Comet. For running experiment updates current step to one from param! Params should be set at the start of the experiment.

Specified by:

logParameter in interface Experiment

Parameters:

parameterName - The name of the param being logged

paramValue - The value for the param being logged

step - The current step for this metric, this will set the given step for this experiment

logHtml

public void logHtml(@NonNull
                    @NonNull String html,
                    boolean override)

Description copied from interface: Experiment

Allows you to create html report for the experiment.

Specified by:

logHtml in interface Experiment

Parameters:

html - A block of html to be sent to Comet

override - Whether previous html sent should be deleted. If true the old html will be deleted. If false, it will have the new html appended onto the end.

logCode

public void logCode(@NonNull
                    @NonNull String code,
                    @NonNull
                    @NonNull String fileName)

Description copied from interface: Experiment

Allows you to report code for the experiment.

Specified by:

logCode in interface Experiment

Parameters:

code - Code to be sent to Comet

fileName - Name of source file to be displayed on UI 'code' tab

logCode

public void logCode(@NonNull
                    @NonNull File asset)

Description copied from interface: Experiment

Allows you to report code for the experiment.

Specified by:

logCode in interface Experiment

Parameters:

asset - Asset with source code to be sent

logOther

public void logOther(@NonNull
                     @NonNull String key,
                     @NonNull
                     @NonNull Object value)

Description copied from interface: Experiment

Logs a key value pair with Comet. This can be used for any sort of generic per experiment data you wish to track.

Specified by:

logOther in interface Experiment

Parameters:

key - The key for the data to be stored

value - The value for said key

addTag

public void addTag(@NonNull
                   @NonNull String tag)

Description copied from interface: Experiment

Adds a tag to this experiment.

Specified by:

addTag in interface Experiment

Parameters:

tag - The tag to be added

logGraph

public void logGraph(@NonNull
                     @NonNull String graph)

Description copied from interface: Experiment

Logs a graph to Comet.

Specified by:

logGraph in interface Experiment

Parameters:

graph - The graph to be logged

logStartTime

public void logStartTime(long startTimeMillis)

Description copied from interface: Experiment

Logs the start time of the experiment.

Specified by:

logStartTime in interface Experiment

Parameters:

startTimeMillis - When you want to say that the experiment started

logEndTime

public void logEndTime(long endTimeMillis)

Description copied from interface: Experiment

Logs the start time of the experiment.

Specified by:

logEndTime in interface Experiment

Parameters:

endTimeMillis - When you want to say that the experiment ended

uploadAsset

public void uploadAsset(@NonNull
                        @NonNull File asset,
                        @NonNull
                        @NonNull String fileName,
                        boolean overwrite,
                        long step,
                        long epoch)

Description copied from interface: Experiment

Upload an asset to be associated with the experiment, for example the trained weights of a neural net. For running experiment updates current step to one from param!

Specified by:

uploadAsset in interface Experiment

Parameters:

asset - The asset to be stored

fileName - The file name under which the asset should be stored in Comet. E.g. "someFile.txt"

overwrite - Whether to overwrite files of the same name in Comet

step - the step to be associated with asset

epoch - the epoch to be associated with asset

uploadAsset

public void uploadAsset(@NonNull
                        @NonNull File asset,
                        boolean overwrite,
                        long step,
                        long epoch)

Specified by:

uploadAsset in interface Experiment

logGitMetadata

public void logGitMetadata(CreateGitMetadata gitMetadata)

Description copied from interface: Experiment

Logs Git Metadata for the experiment.

Specified by:

logGitMetadata in interface Experiment

Parameters:

gitMetadata - The Git Metadata for the experiment

getMetadata

public ExperimentMetadataRest getMetadata()

Description copied from interface: Experiment

Get experiment metadata.

Specified by:

getMetadata in interface Experiment

Returns:

the metadata associated with experiment.

getGitMetadata

public GitMetadataRest getGitMetadata()

Description copied from interface: Experiment

Get experiment git metadata.

Specified by:

getGitMetadata in interface Experiment

Returns:

the GIT metadata for this experiment.

getHtml

public Optional<String> getHtml()

Description copied from interface: Experiment

Get experiment html.

Specified by:

getHtml in interface Experiment

Returns:

the HTML of the experiment.

getOutput

public Optional<String> getOutput()

Description copied from interface: Experiment

Get experiment output data.

Specified by:

getOutput in interface Experiment

Returns:

the output data of the experiment.

getGraph

public Optional<String> getGraph()

Description copied from interface: Experiment

Get experiment graph data.

Specified by:

getGraph in interface Experiment

Returns:

the graph data associated with experiment.

getParameters

public List<ValueMinMaxDto> getParameters()

Description copied from interface: Experiment

Get experiment parameters.

Specified by:

getParameters in interface Experiment

Returns:

the parameters logged by experiment.

getMetrics

public List<ValueMinMaxDto> getMetrics()

Description copied from interface: Experiment

Get experiment metrics.

Specified by:

getMetrics in interface Experiment

Returns:

the metrics logged by experiment.

getLogOther

public List<ValueMinMaxDto> getLogOther()

Description copied from interface: Experiment

Get experiment log other data.

Specified by:

getLogOther in interface Experiment

Returns:

the other data logged with experiment.

getTags

public List<String> getTags()

Description copied from interface: Experiment

Get experiment tags.

Specified by:

getTags in interface Experiment

Returns:

the TAGs associated with experiment

getAssetList

public List<ExperimentAssetLink> getAssetList(@NonNull
                                              @NonNull AssetType type)

Description copied from interface: Experiment

Get experiment asset list.

Specified by:

getAssetList in interface Experiment

Parameters:

type - the type of assets to be included.

Returns:

the list of assets associated with experiment.