Package

ml.combust.bundle

dsl

Permalink

package dsl

Domain specific language for generating Bundle.ML models.

DSL provides objects for generating various Bundle.ML objects.

ml.combust.bundle.dsl.Bundle - Stores the name, version, serialization format, list of attributes, and list of root nodes in the bundle graph.

ml.combust.bundle.dsl.Node - A node for transforming data in the model graph. Nodes have a unique name within a Bundle.ML graph and are responsible for piping inputs to a model and outputs from a model to the rest of the graph via sockets. Sockets map a field name in the graph to a specific port in a model.

A model can have input ports and output ports. An example would be a StringIndexer node named "my_string_indexer" that indexes the input field "my_string_label" to the output field "my_double_label". There are two sockets for this node, one that maps the "my_string_label" field to the "input" port of the model and another socket that maps the "output" port of the model to the "my_double_label" field.

ml.combust.bundle.dsl.Model - A model operation that transforms data within the graph. Models have an op name that define the type of model. For instance, the op name could be "string_indexer", "linear_regression", or "random_forest_regression". In addition to the op name of the model, all of the required attributes, such as coefficients and intercepts ,for executing the model are stored here.

ml.combust.bundle.dsl.Attribute - Attributes are used to store values for ml.combust.bundle.dsl.Models and ml.combust.bundle.dsl.Bundles. They contain a name, a ml.bundle.DataType.DataType and a ml.combust.bundle.dsl.Value.

ml.combust.bundle.dsl.Value - Values store concrete data for an ml.combust.bundle.dsl.Attribute. You can store basic data like strings, doubles, longs and booleans, tensors of any dimension, custom data types, and lists of any depth and data type.

ml.combust.bundle.dsl.AttributeList - Stores a set of ml.combust.bundle.dsl.Attributes.

ml.combust.bundle.dsl.Shape - Stores the map of graph input/output fields to model inputs and outputs.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. dsl
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. case class Attribute(name: String, value: Value) extends Product with Serializable

    Permalink

    Attribute class stores a named value.

    Attribute class stores a named value.

    name

    name of the value

    value

    stored value of the attribute

  2. case class AttributeList(lookup: Map[String, Attribute]) extends WritableAttributeList with Product with Serializable

    Permalink

    Class that holds a list of Attribute objects.

    Class that holds a list of Attribute objects.

    Can only have one attribute with a given name at a time.

    lookup

    map of attribute name to Attribute object

  3. case class Bundle(name: String, format: SerializationFormat, version: String, attributes: Option[AttributeList], nodes: Seq[Any]) extends HasAttributeList[Bundle] with Product with Serializable

    Permalink

    Root object for serializing Bundle.ML pipelines and graphs.

    Root object for serializing Bundle.ML pipelines and graphs.

    name

    name of the bundle

    format

    serialization format of the Bundle

    version

    Bundle.ML version used for serializing

    attributes

    optional AttributeList to serialize with the bundle

    nodes

    list of root nodes in the bundle

  4. trait HasAttributeList[T] extends AnyRef

    Permalink

    This trait provides easy access to reading/writing attributes to objects that contain an AttributeList.

    This trait provides easy access to reading/writing attributes to objects that contain an AttributeList.

    T

    class type that contains the attribute list

  5. case class Model(op: String, attributes: Option[AttributeList] = None) extends WritableModel with HasAttributeList[Model] with Product with Serializable

    Permalink

    Class that encodes all information need to serialize or deserialize a machine learning model.

    Class that encodes all information need to serialize or deserialize a machine learning model.

    Models encode things like coefficients for linear regressions, labels for string indexers, sizes for one hot encoders, decision trees, and any other data needed to serialize and deserialize ML models and feature builders.

    Usually you will want to pass ReadableModel or WritableModel to a user depending on the use case to constrict the allowed operations.

    op

    op name for the model

    attributes

    optional list of attributes for the model

  6. case class Node(name: String, shape: Shape) extends WritableNode with Product with Serializable

    Permalink

    Class for storing a node in the Bundle.ML graph.

    Class for storing a node in the Bundle.ML graph.

    Bundle.ML is composed of a set of Node objects, each with their own inputs and outputs defined by their Node#shape.

    Every Node needs a unique identifier within a Bundle.ML graph. This is the Node#name.

    name

    unique identifier for the node

    shape

    shape of the node

  7. trait ReadableAttributeList extends AnyRef

    Permalink

    Trait for read-only operations of an AttributeList.

    Trait for read-only operations of an AttributeList.

    Use this trait when you only want to grant read-access to an AttributeList. This is used when deserializing Model or Bundle objects.

  8. trait ReadableModel extends AnyRef

    Permalink

    Trait for read-only operations on a Model.

    Trait for read-only operations on a Model.

    Use this trait to expose a read-only interface to a Model. This is primarily used when deserializing.

  9. trait ReadableNode extends AnyRef

    Permalink

    Trait for read-only interface to a Node.

    Trait for read-only interface to a Node.

    Use this trait when deserializing node objects.

  10. trait ReadableShape extends AnyRef

    Permalink

    Trait for read-only interface to Shape objects.

    Trait for read-only interface to Shape objects.

    Use this trait when desierializing Shape objects, such as when reading a Node from a Bundle.ML file.

  11. case class Shape(_shape: bundle.Shape.Shape = ...) extends WritableShape with Product with Serializable

    Permalink

    Class for holding the input fields and output fields of a Node.

    Class for holding the input fields and output fields of a Node. The shape also holds information for connecting the input/output fields to the underlying ML model.

    A Shape contains input and output sockets. Sockets map field data to certain functionality within a Model. For instance, say we want to run a "label" field through a string indexer and have the result output to the field "label_name". We could wire up the node like so:

    scala> import ml.bundle.dsl._
    scala> Shape().withInput("label", "input"). // connect the "label" field to the model input
             withOutput("label_name", "output") // connect the model output to the "label_name" field

    Or more concisely:

    scala> import ml.bundle.dsl._
    scala> Shape().withStandardIO("label", "label_name") // shorthand for the above code
    _shape

    protobuf shape object containing the shape information

  12. case class Value(_bundleDataType: DataType, value: Any) extends Product with Serializable

    Permalink

    This class is used to wrap Scala objects for later serialization into Bundle.ML

    This class is used to wrap Scala objects for later serialization into Bundle.ML

    _bundleDataType

    data type of the value being stored

    value

    Scala object that will be serialized later

  13. trait WritableAttributeList extends ReadableAttributeList

    Permalink

    Trait for write operations on an AttributeList.

    Trait for write operations on an AttributeList.

    Use this trait when you need to provide a writable interface to the AttributeList. This is mainly used when serializing Model or Bundle objects.

  14. trait WritableModel extends ReadableModel

    Permalink

    Trait for a writable interface to a Model.

    Trait for a writable interface to a Model.

    Use this trait during serialization of Model objects.

  15. trait WritableNode extends ReadableNode

    Permalink

    Trait for writable interface to a Node.

    Trait for writable interface to a Node.

    There are no write operations for a Node. This trait is just here for completeness.

  16. trait WritableShape extends ReadableShape

    Permalink

    Trait for writable interface to a Shape.

    Trait for writable interface to a Shape.

    Use this trait when serializing a Shape object, such as when writing a Node to a Bundle.ML file.

Value Members

  1. object Attribute extends Serializable

    Permalink

    Companion object for Attribute class.

  2. object AttributeList extends Serializable

    Permalink

    Companion class for construction and conversion of AttributeList objects.

  3. object Bundle extends Serializable

    Permalink

    Companion class for constants and constructors of Bundle objects.

    Companion class for constants and constructors of Bundle objects.

    Contains file names for bundle JSON files and model JSON files.

  4. object Shape extends Serializable

    Permalink

    Companion object for holding constant values.

  5. object Value extends Serializable

    Permalink

    Provides a set of helper methods for easily creating ml.combust.bundle.dsl.Value objects.

    Provides a set of helper methods for easily creating ml.combust.bundle.dsl.Value objects.

    Easily create ml.combust.bundle.dsl.Value objects of any type using the helper methods provided here. The helper methods will wrap a Scala value for later serializing into Bundle.ML.

    Also provides several helper methods for converting from Bundle.ML protobuf objects back into wrapped Scala objects.

Inherited from AnyRef

Inherited from Any

Ungrouped