Class

org.apache.spark.ml.odkl

GridSearch

Related Doc: package odkl

Permalink

class GridSearch[ModelIn <: ModelWithSummary[ModelIn]] extends ForkedEstimator[ModelIn, ConfigHolder, ModelIn]

Provides ability to search through multiple configurations in parallel mode, collecting all the stat and metrics.

Supports persisting temporary models in order to restore after failures, but only when used with StableOrderParamGridBuilder.

Linear Supertypes
ForkedEstimator[ModelIn, ConfigHolder, ModelIn], ForkedModelParams, SummarizableEstimator[ModelIn], Estimator[ModelIn], PipelineStage, Logging, Params, Serializable, Serializable, Identifiable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. GridSearch
  2. ForkedEstimator
  3. ForkedModelParams
  4. SummarizableEstimator
  5. Estimator
  6. PipelineStage
  7. Logging
  8. Params
  9. Serializable
  10. Serializable
  11. Identifiable
  12. AnyRef
  13. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new GridSearch(nested: SummarizableEstimator[ModelIn])

    Permalink

  2. new GridSearch(nested: SummarizableEstimator[ModelIn], uid: String)

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def $[T](param: Param[T]): T

    Permalink
    Attributes
    protected
    Definition Classes
    Params

  4. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  5. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  6. final val cacheForks: Param[Boolean]

    Permalink
    Definition Classes
    ForkedEstimator

  7. final def clear(param: Param[_]): GridSearch.this.type

    Permalink
    Definition Classes
    Params

  8. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  9. val configurationIndexColumn: Param[String]

    Permalink

  10. val configurations: Block

    Permalink

  11. def copy(extra: ParamMap): SummarizableEstimator[ModelIn]

    Permalink
    Definition Classes
    GridSearch → SummarizableEstimator → Estimator → PipelineStage → Params

  12. def copyValues[T <: Params](to: T, extra: ParamMap): T

    Permalink
    Attributes
    protected
    Definition Classes
    Params

  13. def createForks(dataset: Dataset[_]): Seq[(ConfigHolder, DataFrame)]

    Permalink

    Override this method and create forks to train from the data.

    Override this method and create forks to train from the data.

    Attributes
    protected
    Definition Classes
    GridSearch → ForkedEstimator

  14. final def defaultCopy[T <: Params](extra: ParamMap): T

    Permalink
    Attributes
    protected
    Definition Classes
    Params

  15. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  16. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  17. val errorColumn: Param[String]

    Permalink

  18. val estimatorParamMaps: Param[Array[ParamMap]]

    Permalink

  19. def explainParam(param: Param[_]): String

    Permalink
    Definition Classes
    Params

  20. def explainParams(): String

    Permalink
    Definition Classes
    Params

  21. final def extractParamMap(): ParamMap

    Permalink
    Definition Classes
    Params

  22. final def extractParamMap(extra: ParamMap): ParamMap

    Permalink
    Definition Classes
    Params

  23. def failFast(key: ConfigHolder, triedIn: Try[ModelIn]): Try[ModelIn]

    Permalink
    Attributes
    protected
    Definition Classes
    GridSearch → ForkedEstimator

  24. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  25. def fit(dataset: Dataset[_]): ModelIn

    Permalink
    Definition Classes
    ForkedEstimator → Estimator

  26. def fit(dataset: Dataset[_], paramMaps: Array[ParamMap]): Seq[ModelIn]

    Permalink
    Definition Classes
    Estimator
    Annotations
    @Since( "2.0.0" )

  27. def fit(dataset: Dataset[_], paramMap: ParamMap): ModelIn

    Permalink
    Definition Classes
    Estimator
    Annotations
    @Since( "2.0.0" )

  28. def fit(dataset: Dataset[_], firstParamPair: ParamPair[_], otherParamPairs: ParamPair[_]*): ModelIn

    Permalink
    Definition Classes
    Estimator
    Annotations
    @Since( "2.0.0" ) @varargs()

  29. def fitFork(estimator: SummarizableEstimator[ModelIn], wholeData: Dataset[_], partialData: (ConfigHolder, DataFrame)): (ConfigHolder, Try[ModelIn])

    Permalink
    Definition Classes
    GridSearch → ForkedEstimator

  30. final def get[T](param: Param[T]): Option[T]

    Permalink
    Definition Classes
    Params

  31. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  32. def getConfigurationIndexColumn: String

    Permalink

  33. final def getDefault[T](param: Param[T]): Option[T]

    Permalink
    Definition Classes
    Params

  34. def getEstimatorParamMaps: Array[ParamMap]

    Permalink

  35. def getMetricsBlock: String

    Permalink

  36. def getMetricsExpression: String

    Permalink

  37. final def getOrDefault[T](param: Param[T]): T

    Permalink
    Definition Classes
    Params

  38. def getParam(paramName: String): Param[Any]

    Permalink
    Definition Classes
    Params

  39. def getResultingMetricColumn: String

    Permalink

  40. final def hasDefault[T](param: Param[T]): Boolean

    Permalink
    Definition Classes
    Params

  41. def hasParam(paramName: String): Boolean

    Permalink
    Definition Classes
    Params

  42. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  43. def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  44. def initializeLogIfNecessary(isInterpreter: Boolean): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  45. final def isDefined(param: Param[_]): Boolean

    Permalink
    Definition Classes
    Params

  46. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  47. final def isSet(param: Param[_]): Boolean

    Permalink
    Definition Classes
    Params

  48. def isTraceEnabled(): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  49. def log: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  50. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  51. def logDebug(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  52. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  53. def logError(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  54. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  55. def logInfo(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  56. def logName: String

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  57. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  58. def logTrace(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  59. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  60. def logWarning(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  61. def mayBePropagateKey(data: DataFrame, key: Any): DataFrame

    Permalink
    Attributes
    protected
    Definition Classes
    ForkedModelParams

  62. def mergeModels(sqlContext: SQLContext, models: Seq[(ConfigHolder, Try[ModelIn])]): ModelIn

    Permalink

    Given models trained for each fork create a combined model.

    Given models trained for each fork create a combined model. This model is the result of the estimator.

    Attributes
    protected
    Definition Classes
    GridSearch → ForkedEstimator

  63. val metricsBlock: Param[String]

    Permalink

  64. val metricsExpression: Param[String]

    Permalink

  65. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  66. val nested: SummarizableEstimator[ModelIn]

    Permalink

    Nested estimator to call for each fork.

    Nested estimator to call for each fork.

    Definition Classes
    ForkedEstimator

  67. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  68. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  69. final val numThreads: IntParam

    Permalink
    Definition Classes
    ForkedEstimator

  70. final val overwriteModels: BooleanParam

    Permalink
    Definition Classes
    ForkedEstimator

  71. val paramNames: Param[Map[Param[_], String]]

    Permalink

  72. lazy val params: Array[Param[_]]

    Permalink
    Definition Classes
    Params

  73. final val pathForTempModels: Param[String]

    Permalink
    Definition Classes
    ForkedEstimator

  74. final val persistingKeyColumns: StringArrayParam

    Permalink
    Definition Classes
    ForkedEstimator

  75. final val propagatedKeyColumn: Param[String]

    Permalink
    Definition Classes
    ForkedModelParams

  76. val resultingMetricColumn: Param[String]

    Permalink

  77. final def set(paramPair: ParamPair[_]): GridSearch.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    Params

  78. final def set(param: String, value: Any): GridSearch.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    Params

  79. final def set[T](param: Param[T], value: T): GridSearch.this.type

    Permalink
    Definition Classes
    Params

  80. def setCacheForks(value: Boolean): GridSearch.this.type

    Permalink
    Definition Classes
    ForkedEstimator

  81. def setConfigurationIndexColumn(value: String): GridSearch.this.type

    Permalink

  82. final def setDefault(paramPairs: ParamPair[_]*): GridSearch.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    Params

  83. final def setDefault[T](param: Param[T], value: T): GridSearch.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    Params

  84. def setEstimatorParamMaps(value: Array[ParamMap]): GridSearch.this.type

    Permalink

  85. def setMetricsBlock(value: String): GridSearch.this.type

    Permalink

  86. def setMetricsExpression(value: String): GridSearch.this.type

    Permalink

  87. def setNumThreads(value: Int): GridSearch.this.type

    Permalink
    Definition Classes
    ForkedEstimator

  88. def setOverwriteModels(value: Boolean): GridSearch.this.type

    Permalink
    Definition Classes
    ForkedEstimator

  89. def setParamNames(value: (Param[_], String)*): GridSearch.this.type

    Permalink

  90. def setPathForTempModels(value: String): GridSearch.this.type

    Permalink
    Definition Classes
    ForkedEstimator

  91. def setPropagatedKeyColumn(value: String): GridSearch.this.type

    Permalink
    Definition Classes
    ForkedModelParams

  92. def setResultingMetricColumn(value: String): GridSearch.this.type

    Permalink

  93. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  94. def toString(): String

    Permalink
    Definition Classes
    Identifiable → AnyRef → Any

  95. def transformSchema(schema: StructType): StructType

    Permalink
    Definition Classes
    ForkedEstimator → PipelineStage
    Annotations
    @DeveloperApi()

  96. def transformSchema(schema: StructType, logging: Boolean): StructType

    Permalink
    Attributes
    protected
    Definition Classes
    PipelineStage
    Annotations
    @DeveloperApi()

  97. val uid: String

    Permalink
    Definition Classes
    GridSearch → ForkedEstimator → Identifiable

  98. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  99. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  100. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from ForkedEstimator[ModelIn, ConfigHolder, ModelIn]

Inherited from ForkedModelParams

Inherited from SummarizableEstimator[ModelIn]

Inherited from Estimator[ModelIn]

Inherited from PipelineStage

Inherited from Logging

Inherited from Params

Inherited from Serializable

Inherited from Serializable

Inherited from Identifiable

Inherited from AnyRef

Inherited from Any

Ungrouped