Type Members

1.  case class Aggregate(groupingExpressions: Seq[Expression], aggregateExpressions: Seq[NamedExpression], child: LogicalPlan) extends UnaryNode with Product with Serializable

   This is a Group by operator with the aggregate functions and projections.

   This is a Group by operator with the aggregate functions and projections.

   groupingExpressions

   expressions for grouping keys

   aggregateExpressions

   expressions for a project list, which could contain AggregateFunctions. Note: Currently, aggregateExpressions is the project list of this Group by operator. Before separating projection from grouping and aggregate, we should avoid expression-level optimization on aggregateExpressions, which could reference an expression in groupingExpressions. For example, see the rule org.apache.spark.sql.catalyst.optimizer.SimplifyExtractValueOps

2.  case class AlterNamespaceSetLocation(namespace: LogicalPlan, location: String) extends LogicalPlan with Command with Product with Serializable

   The logical plan of the ALTER (DATABASE|SCHEMA|NAMESPACE) ...

   The logical plan of the ALTER (DATABASE|SCHEMA|NAMESPACE) ... SET LOCATION command that works for v2 catalogs.

3.  case class AlterNamespaceSetProperties(namespace: LogicalPlan, properties: Map[String, String]) extends LogicalPlan with Command with Product with Serializable

   The logical plan of the ALTER (DATABASE|SCHEMA|NAMESPACE) ...

   The logical plan of the ALTER (DATABASE|SCHEMA|NAMESPACE) ... SET (DBPROPERTIES|PROPERTIES) command that works for v2 catalogs.

4.  case class AlterTable(catalog: TableCatalog, ident: Identifier, table: NamedRelation, changes: Seq[TableChange]) extends LogicalPlan with Command with Product with Serializable

   The logical plan of the ALTER TABLE command that works for v2 tables.

5.  case class AlterTableAddColumnsStatement(tableName: Seq[String], columnsToAdd: Seq[QualifiedColType]) extends ParsedStatement with Product with Serializable

   ALTER TABLE ...

   ALTER TABLE ... ADD COLUMNS command, as parsed from SQL.

6.  case class AlterTableAddPartitionStatement(tableName: Seq[String], partitionSpecsAndLocs: Seq[(TablePartitionSpec, Option[String])], ifNotExists: Boolean) extends ParsedStatement with Product with Serializable

   ALTER TABLE ...

   ALTER TABLE ... ADD PARTITION command, as parsed from SQL

7.  case class AlterTableAlterColumnStatement(tableName: Seq[String], column: Seq[String], dataType: Option[DataType], nullable: Option[Boolean], comment: Option[String], position: Option[ColumnPosition]) extends ParsedStatement with Product with Serializable

   ALTER TABLE ...

   ALTER TABLE ... CHANGE COLUMN command, as parsed from SQL.

8.  case class AlterTableDropColumnsStatement(tableName: Seq[String], columnsToDrop: Seq[Seq[String]]) extends ParsedStatement with Product with Serializable

   ALTER TABLE ...

   ALTER TABLE ... DROP COLUMNS command, as parsed from SQL.

9.  case class AlterTableDropPartitionStatement(tableName: Seq[String], specs: Seq[TablePartitionSpec], ifExists: Boolean, purge: Boolean, retainData: Boolean) extends ParsedStatement with Product with Serializable

   ALTER TABLE ...

   ALTER TABLE ... DROP PARTITION command, as parsed from SQL

10.  case class AlterTableRecoverPartitionsStatement(tableName: Seq[String]) extends ParsedStatement with Product with Serializable

    ALTER TABLE ...

    ALTER TABLE ... RECOVER PARTITIONS command, as parsed from SQL.

11.  case class AlterTableRenameColumnStatement(tableName: Seq[String], column: Seq[String], newName: String) extends ParsedStatement with Product with Serializable

    ALTER TABLE ...

    ALTER TABLE ... RENAME COLUMN command, as parsed from SQL.

12.  case class AlterTableRenamePartitionStatement(tableName: Seq[String], from: TablePartitionSpec, to: TablePartitionSpec) extends ParsedStatement with Product with Serializable

    ALTER TABLE ...

    ALTER TABLE ... RENAME PARTITION command, as parsed from SQL.

13.  case class AlterTableSerDePropertiesStatement(tableName: Seq[String], serdeClassName: Option[String], serdeProperties: Option[Map[String, String]], partitionSpec: Option[TablePartitionSpec]) extends ParsedStatement with Product with Serializable

    ALTER TABLE ...

    ALTER TABLE ... SERDEPROPERTIES command, as parsed from SQL

14.  case class AlterTableSetLocationStatement(tableName: Seq[String], partitionSpec: Option[TablePartitionSpec], location: String) extends ParsedStatement with Product with Serializable

    ALTER TABLE ...

    ALTER TABLE ... SET LOCATION command, as parsed from SQL.

15.  case class AlterTableSetPropertiesStatement(tableName: Seq[String], properties: Map[String, String]) extends ParsedStatement with Product with Serializable

    ALTER TABLE ...

    ALTER TABLE ... SET TBLPROPERTIES command, as parsed from SQL.

16.  case class AlterTableUnsetPropertiesStatement(tableName: Seq[String], propertyKeys: Seq[String], ifExists: Boolean) extends ParsedStatement with Product with Serializable

    ALTER TABLE ...

    ALTER TABLE ... UNSET TBLPROPERTIES command, as parsed from SQL.

17.  case class AlterViewAsStatement(viewName: Seq[String], originalText: String, query: LogicalPlan) extends ParsedStatement with Product with Serializable

    ALTER VIEW ...

    ALTER VIEW ... Query command, as parsed from SQL.

18.  case class AlterViewSetPropertiesStatement(viewName: Seq[String], properties: Map[String, String]) extends ParsedStatement with Product with Serializable

    ALTER VIEW ...

    ALTER VIEW ... SET TBLPROPERTIES command, as parsed from SQL.

19.  case class AlterViewUnsetPropertiesStatement(viewName: Seq[String], propertyKeys: Seq[String], ifExists: Boolean) extends ParsedStatement with Product with Serializable

    ALTER VIEW ...

    ALTER VIEW ... UNSET TBLPROPERTIES command, as parsed from SQL.

20.  trait AnalysisHelper extends QueryPlan[LogicalPlan]

    AnalysisHelper defines some infrastructure for the query analyzer.

    AnalysisHelper defines some infrastructure for the query analyzer. In particular, in query analysis we don't want to repeatedly re-analyze sub-plans that have previously been analyzed.

    This trait defines a flag analyzed that can be set to true once analysis is done on the tree. This also provides a set of resolve methods that do not recurse down to sub-plans that have the analyzed flag set to true.

    The analyzer rules should use the various resolve methods, in lieu of the various transform methods defined in TreeNode and QueryPlan.

    To prevent accidental use of the transform methods, this trait also overrides the transform methods to throw exceptions in test mode, if they are used in the analyzer.

21.  case class AnalyzeColumnStatement(tableName: Seq[String], columnNames: Option[Seq[String]], allColumns: Boolean) extends ParsedStatement with Product with Serializable

    An ANALYZE TABLE FOR COLUMNS statement, as parsed from SQL.

22.  case class AnalyzeTableStatement(tableName: Seq[String], partitionSpec: Map[String, Option[String]], noScan: Boolean) extends ParsedStatement with Product with Serializable

    An ANALYZE TABLE statement, as parsed from SQL.

23.  case class AppendColumns(func: (Any) ⇒ Any, argumentClass: Class[_], argumentSchema: StructType, deserializer: Expression, serializer: Seq[NamedExpression], child: LogicalPlan) extends UnaryNode with Product with Serializable

    A relation produced by applying func to each element of the child, concatenating the resulting columns at the end of the input row.

    A relation produced by applying func to each element of the child, concatenating the resulting columns at the end of the input row.

    deserializer

    used to extract the input to func from an input row.

    serializer

    use to serialize the output of func.

24.  case class AppendColumnsWithObject(func: (Any) ⇒ Any, childSerializer: Seq[NamedExpression], newColumnsSerializer: Seq[NamedExpression], child: LogicalPlan) extends UnaryNode with ObjectConsumer with Product with Serializable

    An optimized version of AppendColumns, that can be executed on deserialized object directly.

25.  case class AppendData(table: NamedRelation, query: LogicalPlan, writeOptions: Map[String, String], isByName: Boolean) extends LogicalPlan with V2WriteCommand with Product with Serializable

    Append data to an existing table.

26.  case class ArrowEvalPython(udfs: Seq[PythonUDF], resultAttrs: Seq[Attribute], child: LogicalPlan, evalType: Int) extends UnaryNode with BaseEvalPython with Product with Serializable

    A logical plan that evaluates a PythonUDF with Apache Arrow.

27.  case class Assignment(key: Expression, value: Expression) extends Expression with Unevaluable with Product with Serializable
28.  trait BaseEvalPython extends UnaryNode
29.  case class BatchEvalPython(udfs: Seq[PythonUDF], resultAttrs: Seq[Attribute], child: LogicalPlan) extends UnaryNode with BaseEvalPython with Product with Serializable

    A logical plan that evaluates a PythonUDF

30.  abstract class BinaryNode extends LogicalPlan

    A logical plan node with a left and right child.

31.  case class CacheTableStatement(tableName: Seq[String], plan: Option[LogicalPlan], isLazy: Boolean, options: Map[String, String]) extends ParsedStatement with Product with Serializable

    A CACHE TABLE statement, as parsed from SQL

32.  case class CoGroup(func: (Any, Iterator[Any], Iterator[Any]) ⇒ TraversableOnce[Any], keyDeserializer: Expression, leftDeserializer: Expression, rightDeserializer: Expression, leftGroup: Seq[Attribute], rightGroup: Seq[Attribute], leftAttr: Seq[Attribute], rightAttr: Seq[Attribute], outputObjAttr: Attribute, left: LogicalPlan, right: LogicalPlan) extends BinaryNode with ObjectProducer with Product with Serializable

    A relation produced by applying func to each grouping key and associated values from left and right children.

33.  case class CollectMetrics(name: String, metrics: Seq[NamedExpression], child: LogicalPlan) extends UnaryNode with Product with Serializable

    Collect arbitrary (named) metrics from a dataset.

    Collect arbitrary (named) metrics from a dataset. As soon as the query reaches a completion point (batch query completes or streaming query epoch completes) an event is emitted on the driver which can be observed by attaching a listener to the spark session. The metrics are named so we can collect metrics at multiple places in a single dataset.

    This node behaves like a global aggregate. All the metrics collected must be aggregate functions or be literals.

34.  case class ColumnStat(distinctCount: Option[BigInt] = None, min: Option[Any] = None, max: Option[Any] = None, nullCount: Option[BigInt] = None, avgLen: Option[Long] = None, maxLen: Option[Long] = None, histogram: Option[Histogram] = None, version: Int = CatalogColumnStat.VERSION) extends Product with Serializable

    Statistics collected for a column.

    Statistics collected for a column.

    1. The JVM data type stored in min/max is the internal data type for the corresponding Catalyst data type. For example, the internal type of DateType is Int, and that the internal type of TimestampType is Long. 2. There is no guarantee that the statistics collected are accurate. Approximation algorithms (sketches) might have been used, and the data collected can also be stale.

    distinctCount

    number of distinct values

    min

    minimum value

    max

    maximum value

    nullCount

    number of nulls

    avgLen

    average length of the values. For fixed-length types, this should be a constant.

    maxLen

    maximum length of the values. For fixed-length types, this should be a constant.

    histogram

    histogram of the values

    version

    version of statistics saved to or retrieved from the catalog

35.  trait Command extends LogicalPlan

    A logical node that represents a non-query command to be executed by the system.

    A logical node that represents a non-query command to be executed by the system. For example, commands can be used by parsers to represent DDL operations. Commands, unlike queries, are eagerly executed.

36.  case class CommentOnNamespace(child: LogicalPlan, comment: String) extends LogicalPlan with Command with Product with Serializable

    The logical plan that defines or changes the comment of an NAMESPACE for v2 catalogs.

    The logical plan that defines or changes the comment of an NAMESPACE for v2 catalogs.

    COMMENT ON (DATABASE|SCHEMA|NAMESPACE) namespaceIdentifier IS ('text' | NULL)

    where the text is the new comment written as a string literal; or NULL to drop the comment.

37.  case class CommentOnTable(child: LogicalPlan, comment: String) extends LogicalPlan with Command with Product with Serializable

    The logical plan that defines or changes the comment of an TABLE for v2 catalogs.

    The logical plan that defines or changes the comment of an TABLE for v2 catalogs.

    COMMENT ON TABLE tableIdentifier IS ('text' | NULL)

    where the text is the new comment written as a string literal; or NULL to drop the comment.

38.  trait ConstraintHelper extends AnyRef
39.  case class CreateFunctionStatement(functionName: Seq[String], className: String, resources: Seq[FunctionResource], isTemp: Boolean, ignoreIfExists: Boolean, replace: Boolean) extends ParsedStatement with Product with Serializable

    CREATE FUNCTION statement, as parsed from SQL

40.  case class CreateNamespace(catalog: SupportsNamespaces, namespace: Seq[String], ifNotExists: Boolean, properties: Map[String, String]) extends LogicalPlan with Command with Product with Serializable

    The logical plan of the CREATE NAMESPACE command that works for v2 catalogs.

41.  case class CreateNamespaceStatement(namespace: Seq[String], ifNotExists: Boolean, properties: Map[String, String]) extends ParsedStatement with Product with Serializable

    A CREATE NAMESPACE statement, as parsed from SQL.

42.  case class CreateTableAsSelect(catalog: TableCatalog, tableName: Identifier, partitioning: Seq[Transform], query: LogicalPlan, properties: Map[String, String], writeOptions: Map[String, String], ignoreIfExists: Boolean) extends LogicalPlan with Command with V2CreateTablePlan with Product with Serializable

    Create a new table from a select query with a v2 catalog.

43.  case class CreateTableAsSelectStatement(tableName: Seq[String], asSelect: LogicalPlan, partitioning: Seq[Transform], bucketSpec: Option[BucketSpec], properties: Map[String, String], provider: Option[String], options: Map[String, String], location: Option[String], comment: Option[String], writeOptions: Map[String, String], ifNotExists: Boolean) extends ParsedStatement with Product with Serializable

    A CREATE TABLE AS SELECT command, as parsed from SQL.

44.  case class CreateTableStatement(tableName: Seq[String], tableSchema: StructType, partitioning: Seq[Transform], bucketSpec: Option[BucketSpec], properties: Map[String, String], provider: Option[String], options: Map[String, String], location: Option[String], comment: Option[String], ifNotExists: Boolean) extends ParsedStatement with Product with Serializable

    A CREATE TABLE command, as parsed from SQL.

    A CREATE TABLE command, as parsed from SQL.

    This is a metadata-only command and is not used to write data to the created table.

45.  case class CreateV2Table(catalog: TableCatalog, tableName: Identifier, tableSchema: StructType, partitioning: Seq[Transform], properties: Map[String, String], ignoreIfExists: Boolean) extends LogicalPlan with Command with V2CreateTablePlan with Product with Serializable

    Create a new table with a v2 catalog.

46.  case class CreateViewStatement(viewName: Seq[String], userSpecifiedColumns: Seq[(String, Option[String])], comment: Option[String], properties: Map[String, String], originalText: Option[String], child: LogicalPlan, allowExisting: Boolean, replace: Boolean, viewType: ViewType) extends ParsedStatement with Product with Serializable

    A CREATE VIEW statement, as parsed from SQL.

47.  case class Deduplicate(keys: Seq[Attribute], child: LogicalPlan) extends UnaryNode with Product with Serializable

    A logical plan for dropDuplicates.

48.  case class DeleteAction(condition: Option[Expression]) extends MergeAction with Product with Serializable
49.  case class DeleteFromTable(table: LogicalPlan, condition: Option[Expression]) extends LogicalPlan with Command with SupportsSubquery with Product with Serializable

    The logical plan of the DELETE FROM command that works for v2 tables.

50.  case class DescribeColumnStatement(tableName: Seq[String], colNameParts: Seq[String], isExtended: Boolean) extends ParsedStatement with Product with Serializable

    A DESCRIBE TABLE tbl_name col_name statement, as parsed from SQL.

51.  case class DescribeFunctionStatement(functionName: Seq[String], isExtended: Boolean) extends ParsedStatement with Product with Serializable

    A DESCRIBE FUNCTION statement, as parsed from SQL

52.  case class DescribeNamespace(namespace: LogicalPlan, extended: Boolean) extends LogicalPlan with Command with Product with Serializable

    The logical plan of the DESCRIBE NAMESPACE command that works for v2 catalogs.

53.  case class DescribeRelation(relation: LogicalPlan, partitionSpec: TablePartitionSpec, isExtended: Boolean) extends LogicalPlan with Command with Product with Serializable

    The logical plan of the DESCRIBE relation_name command that works for v2 tables.

54.  case class DeserializeToObject(deserializer: Expression, outputObjAttr: Attribute, child: LogicalPlan) extends UnaryNode with ObjectProducer with Product with Serializable

    Takes the input row from child and turns it into object using the given deserializer expression.

55.  case class Distinct(child: LogicalPlan) extends UnaryNode with Product with Serializable

    Returns a new logical plan that dedups input rows.

56.  case class DropFunctionStatement(functionName: Seq[String], ifExists: Boolean, isTemp: Boolean) extends ParsedStatement with Product with Serializable

    DROP FUNCTION statement, as parsed from SQL

57.  case class DropNamespace(namespace: LogicalPlan, ifExists: Boolean, cascade: Boolean) extends LogicalPlan with Command with Product with Serializable

    The logical plan of the DROP NAMESPACE command that works for v2 catalogs.

58.  case class DropTable(catalog: TableCatalog, ident: Identifier, ifExists: Boolean) extends LogicalPlan with Command with Product with Serializable

    The logical plan of the DROP TABLE command that works for v2 tables.

59.  case class DropTableStatement(tableName: Seq[String], ifExists: Boolean, purge: Boolean) extends ParsedStatement with Product with Serializable

    A DROP TABLE statement, as parsed from SQL.

60.  case class DropViewStatement(viewName: Seq[String], ifExists: Boolean) extends ParsedStatement with Product with Serializable

    A DROP VIEW statement, as parsed from SQL.

61.  case class EventTimeWatermark(eventTime: Attribute, delay: CalendarInterval, child: LogicalPlan) extends UnaryNode with Product with Serializable

    Used to mark a user specified column as holding the event time for a row.

62.  case class Except(left: LogicalPlan, right: LogicalPlan, isAll: Boolean) extends SetOperation with Product with Serializable
63.  case class Expand(projections: Seq[Seq[Expression]], output: Seq[Attribute], child: LogicalPlan) extends UnaryNode with Product with Serializable

    Apply a number of projections to every input row, hence we will get multiple output rows for an input row.

    Apply a number of projections to every input row, hence we will get multiple output rows for an input row.

    projections

    to apply

    output

    of all projections.

    child

    operator.

64.  case class Filter(condition: Expression, child: LogicalPlan) extends OrderPreservingUnaryNode with PredicateHelper with Product with Serializable
65.  case class FlatMapCoGroupsInPandas(leftAttributes: Seq[Attribute], rightAttributes: Seq[Attribute], functionExpr: Expression, output: Seq[Attribute], left: LogicalPlan, right: LogicalPlan) extends BinaryNode with Product with Serializable

    Flatmap cogroups using a udf: pandas.Dataframe, pandas.Dataframe -> pandas.Dataframe This is used by DataFrame.groupby().cogroup().apply().

66.  case class FlatMapGroupsInPandas(groupingAttributes: Seq[Attribute], functionExpr: Expression, output: Seq[Attribute], child: LogicalPlan) extends UnaryNode with Product with Serializable

    FlatMap groups using a udf: pandas.Dataframe -> pandas.DataFrame.

    FlatMap groups using a udf: pandas.Dataframe -> pandas.DataFrame. This is used by DataFrame.groupby().apply().

67.  case class FlatMapGroupsInR(func: Array[Byte], packageNames: Array[Byte], broadcastVars: Array[Broadcast[AnyRef]], inputSchema: StructType, outputSchema: StructType, keyDeserializer: Expression, valueDeserializer: Expression, groupingAttributes: Seq[Attribute], dataAttributes: Seq[Attribute], outputObjAttr: Attribute, child: LogicalPlan) extends UnaryNode with ObjectProducer with Product with Serializable
68.  case class FlatMapGroupsInRWithArrow(func: Array[Byte], packageNames: Array[Byte], broadcastVars: Array[Broadcast[AnyRef]], inputSchema: StructType, output: Seq[Attribute], keyDeserializer: Expression, groupingAttributes: Seq[Attribute], child: LogicalPlan) extends UnaryNode with Product with Serializable

    Similar with FlatMapGroupsInR but serializes and deserializes input/output in Arrow format.

    Similar with FlatMapGroupsInR but serializes and deserializes input/output in Arrow format. This is also somewhat similar with FlatMapGroupsInPandas.

69.  case class FlatMapGroupsWithState(func: (Any, Iterator[Any], LogicalGroupState[Any]) ⇒ Iterator[Any], keyDeserializer: Expression, valueDeserializer: Expression, groupingAttributes: Seq[Attribute], dataAttributes: Seq[Attribute], outputObjAttr: Attribute, stateEncoder: ExpressionEncoder[Any], outputMode: OutputMode, isMapGroupsWithState: Boolean = false, timeout: GroupStateTimeout, child: LogicalPlan) extends UnaryNode with ObjectProducer with Product with Serializable

    Applies func to each unique group in child, based on the evaluation of groupingAttributes, while using state data.

    Applies func to each unique group in child, based on the evaluation of groupingAttributes, while using state data. Func is invoked with an object representation of the grouping key an iterator containing the object representation of all the rows with that key.

    func

    function called on each group

    keyDeserializer

    used to extract the key object for each group.

    valueDeserializer

    used to extract the items in the iterator from an input row.

    groupingAttributes

    used to group the data

    dataAttributes

    used to read the data

    outputObjAttr

    used to define the output object

    stateEncoder

    used to serialize/deserialize state before calling func

    outputMode

    the output mode of func

    isMapGroupsWithState

    whether it is created by the mapGroupsWithState method

    timeout

    used to timeout groups that have not received data in a while

70.  case class Generate(generator: Generator, unrequiredChildIndex: Seq[Int], outer: Boolean, qualifier: Option[String], generatorOutput: Seq[Attribute], child: LogicalPlan) extends UnaryNode with Product with Serializable

    Applies a Generator to a stream of input rows, combining the output of each into a new stream of rows.

    Applies a Generator to a stream of input rows, combining the output of each into a new stream of rows. This operation is similar to a flatMap in functional programming with one important additional feature, which allows the input rows to be joined with their output.

    generator

    the generator expression

    unrequiredChildIndex

    this parameter starts as Nil and gets filled by the Optimizer. It's used as an optimization for omitting data generation that will be discarded next by a projection. A common use case is when we explode(array(..)) and are interested only in the exploded data and not in the original array. before this optimization the array got duplicated for each of its elements, causing O(n^^2) memory consumption. (see [SPARK-21657])

    outer

    when true, each input row will be output at least once, even if the output of the given generator is empty.

    qualifier

    Qualifier for the attributes of generator(UDTF)

    generatorOutput

    The output schema of the Generator.

    child

    Children logical plan node

71.  case class GlobalLimit(limitExpr: Expression, child: LogicalPlan) extends OrderPreservingUnaryNode with Product with Serializable

    A global (coordinated) limit.

    A global (coordinated) limit. This operator can emit at most limitExpr number in total.

    See Limit for more information.

72.  case class GroupingSets(selectedGroupByExprs: Seq[Seq[Expression]], groupByExprs: Seq[Expression], child: LogicalPlan, aggregations: Seq[NamedExpression]) extends UnaryNode with Product with Serializable

    A GROUP BY clause with GROUPING SETS can generate a result set equivalent to generated by a UNION ALL of multiple simple GROUP BY clauses.

    A GROUP BY clause with GROUPING SETS can generate a result set equivalent to generated by a UNION ALL of multiple simple GROUP BY clauses.

    We will transform GROUPING SETS into logical plan Aggregate(.., Expand) in Analyzer

    selectedGroupByExprs

    A sequence of selected GroupBy expressions, all exprs should exist in groupByExprs.

    groupByExprs

    The Group By expressions candidates.

    child

    Child operator

    aggregations

    The Aggregation expressions, those non selected group by expressions will be considered as constant null if it appears in the expressions

73.  trait HintErrorHandler extends AnyRef

    The callback for implementing customized strategies of handling hint errors.

74.  case class HintInfo(strategy: Option[JoinStrategyHint] = None) extends Product with Serializable

    The hint attributes to be applied on a specific node.

    The hint attributes to be applied on a specific node.

    strategy

    The preferred join strategy.

75.  case class Histogram(height: Double, bins: Array[HistogramBin]) extends Product with Serializable

    This class is an implementation of equi-height histogram.

    This class is an implementation of equi-height histogram. Equi-height histogram represents the distribution of a column's values by a sequence of bins. Each bin has a value range and contains approximately the same number of rows.

    height

    number of rows in each bin

    bins

    equi-height histogram bins

76.  case class HistogramBin(lo: Double, hi: Double, ndv: Long) extends Product with Serializable

    A bin in an equi-height histogram.

    A bin in an equi-height histogram. We use double type for lower/higher bound for simplicity.

    lo

    lower bound of the value range in this bin

    hi

    higher bound of the value range in this bin

    ndv

    approximate number of distinct values in this bin

77.  trait IgnoreCachedData extends LogicalPlan

    A LogicalPlan operator that does not use the cached results stored in CacheManager

78.  case class InsertAction(condition: Option[Expression], assignments: Seq[Assignment]) extends MergeAction with Product with Serializable
79.  case class InsertIntoDir(isLocal: Boolean, storage: CatalogStorageFormat, provider: Option[String], child: LogicalPlan, overwrite: Boolean = true) extends UnaryNode with Product with Serializable

    Insert query result into a directory.

    Insert query result into a directory.

    isLocal

    Indicates whether the specified directory is local directory

    storage

    Info about output file, row and what serialization format

    provider

    Specifies what data source to use; only used for data source file.

    child

    The query to be executed

    overwrite

    If true, the existing directory will be overwritten Note that this plan is unresolved and has to be replaced by the concrete implementations during analysis.

80.  case class InsertIntoStatement(table: LogicalPlan, partitionSpec: Map[String, Option[String]], query: LogicalPlan, overwrite: Boolean, ifPartitionNotExists: Boolean) extends ParsedStatement with Product with Serializable

    An INSERT INTO statement, as parsed from SQL.

    An INSERT INTO statement, as parsed from SQL.

    table

    the logical plan representing the table.

    partitionSpec

    a map from the partition key to the partition value (optional). If the value is missing, dynamic partition insert will be performed. As an example, INSERT INTO tbl PARTITION (a=1, b=2) AS would have Map('a' -> Some('1'), 'b' -> Some('2')), and INSERT INTO tbl PARTITION (a=1, b) AS ... would have Map('a' -> Some('1'), 'b' -> None).

    query

    the logical plan representing data to write to.

    overwrite

    overwrite existing table or partitions.

    ifPartitionNotExists

    If true, only write if the partition does not exist. Only valid for static partitions.

81.  case class Intersect(left: LogicalPlan, right: LogicalPlan, isAll: Boolean) extends SetOperation with Product with Serializable
82.  case class Join(left: LogicalPlan, right: LogicalPlan, joinType: JoinType, condition: Option[Expression], hint: JoinHint) extends BinaryNode with PredicateHelper with Product with Serializable
83.  case class JoinHint(leftHint: Option[HintInfo], rightHint: Option[HintInfo]) extends Product with Serializable

    Hint that is associated with a Join node, with HintInfo on its left child and on its right child respectively.

84.  sealed abstract class JoinStrategyHint extends AnyRef
85.  abstract class LeafNode extends LogicalPlan

    A logical plan node with no children.

86.  case class LoadDataStatement(tableName: Seq[String], path: String, isLocal: Boolean, isOverwrite: Boolean, partition: Option[TablePartitionSpec]) extends ParsedStatement with Product with Serializable

    A LOAD DATA INTO TABLE statement, as parsed from SQL

87.  case class LocalLimit(limitExpr: Expression, child: LogicalPlan) extends OrderPreservingUnaryNode with Product with Serializable

    A partition-local (non-coordinated) limit.

    A partition-local (non-coordinated) limit. This operator can emit at most limitExpr number of tuples on each physical partition.

    See Limit for more information.

88.  case class LocalRelation(output: Seq[Attribute], data: Seq[InternalRow] = Nil, isStreaming: Boolean = false) extends LeafNode with MultiInstanceRelation with Product with Serializable

    Logical plan node for scanning data from a local collection.

    Logical plan node for scanning data from a local collection.

    data

    The local collection holding the data. It doesn't need to be sent to executors and then doesn't need to be serializable.

89.  trait LogicalGroupState[S] extends AnyRef

    Internal class representing State

90.  abstract class LogicalPlan extends QueryPlan[LogicalPlan] with AnalysisHelper with LogicalPlanStats with QueryPlanConstraints with Logging
91.  trait LogicalPlanVisitor[T] extends AnyRef

    A visitor pattern for traversing a LogicalPlan tree and computing some properties.

92.  case class MapElements(func: AnyRef, argumentClass: Class[_], argumentSchema: StructType, outputObjAttr: Attribute, child: LogicalPlan) extends UnaryNode with ObjectConsumer with ObjectProducer with Product with Serializable

    A relation produced by applying func to each element of the child.

93.  case class MapGroups(func: (Any, Iterator[Any]) ⇒ TraversableOnce[Any], keyDeserializer: Expression, valueDeserializer: Expression, groupingAttributes: Seq[Attribute], dataAttributes: Seq[Attribute], outputObjAttr: Attribute, child: LogicalPlan) extends UnaryNode with ObjectProducer with Product with Serializable

    Applies func to each unique group in child, based on the evaluation of groupingAttributes.

    Applies func to each unique group in child, based on the evaluation of groupingAttributes. Func is invoked with an object representation of the grouping key an iterator containing the object representation of all the rows with that key.

    keyDeserializer

    used to extract the key object for each group.

    valueDeserializer

    used to extract the items in the iterator from an input row.

94.  case class MapInPandas(functionExpr: Expression, output: Seq[Attribute], child: LogicalPlan) extends UnaryNode with Product with Serializable

    Map partitions using a udf: iter(pandas.Dataframe) -> iter(pandas.DataFrame).

    Map partitions using a udf: iter(pandas.Dataframe) -> iter(pandas.DataFrame). This is used by DataFrame.mapInPandas()

95.  case class MapPartitions(func: (Iterator[Any]) ⇒ Iterator[Any], outputObjAttr: Attribute, child: LogicalPlan) extends UnaryNode with ObjectConsumer with ObjectProducer with Product with Serializable

    A relation produced by applying func to each partition of the child.

96.  case class MapPartitionsInR(func: Array[Byte], packageNames: Array[Byte], broadcastVars: Array[Broadcast[AnyRef]], inputSchema: StructType, outputSchema: StructType, outputObjAttr: Attribute, child: LogicalPlan) extends UnaryNode with ObjectConsumer with ObjectProducer with Product with Serializable

    A relation produced by applying a serialized R function func to each partition of the child.

97.  case class MapPartitionsInRWithArrow(func: Array[Byte], packageNames: Array[Byte], broadcastVars: Array[Broadcast[AnyRef]], inputSchema: StructType, output: Seq[Attribute], child: LogicalPlan) extends UnaryNode with Product with Serializable

    Similar with MapPartitionsInR but serializes and deserializes input/output in Arrow format.

    Similar with MapPartitionsInR but serializes and deserializes input/output in Arrow format.

    This is somewhat similar with org.apache.spark.sql.execution.python.ArrowEvalPython

98.  sealed abstract class MergeAction extends Expression with Unevaluable
99.  case class MergeIntoTable(targetTable: LogicalPlan, sourceTable: LogicalPlan, mergeCondition: Expression, matchedActions: Seq[MergeAction], notMatchedActions: Seq[MergeAction]) extends LogicalPlan with Command with SupportsSubquery with Product with Serializable

    The logical plan of the MERGE INTO command that works for v2 tables.

100.  trait ObjectConsumer extends UnaryNode

     A trait for logical operators that consumes domain objects as input.

     A trait for logical operators that consumes domain objects as input. The output of its child must be a single-field row containing the input object.

101.  trait ObjectProducer extends LogicalPlan

     A trait for logical operators that produces domain objects as output.

     A trait for logical operators that produces domain objects as output. The output of this operator is a single-field safe row containing the produced object.

102.  case class OneRowRelation() extends LeafNode with Product with Serializable

     A relation with one row.

     A relation with one row. This is used in "SELECT ..." without a from clause.

103.  abstract class OrderPreservingUnaryNode extends UnaryNode
104.  case class OverwriteByExpression(table: NamedRelation, deleteExpr: Expression, query: LogicalPlan, writeOptions: Map[String, String], isByName: Boolean) extends LogicalPlan with V2WriteCommand with Product with Serializable

     Overwrite data matching a filter in an existing table.

105.  case class OverwritePartitionsDynamic(table: NamedRelation, query: LogicalPlan, writeOptions: Map[String, String], isByName: Boolean) extends LogicalPlan with V2WriteCommand with Product with Serializable

     Dynamically overwrite partitions in an existing table.

106.  abstract class ParsedStatement extends LogicalPlan

     A logical plan node that contains exactly what was parsed from SQL.

     A logical plan node that contains exactly what was parsed from SQL.

     This is used to hold information parsed from SQL when there are multiple implementations of a query or command. For example, CREATE TABLE may be implemented by different nodes for v1 and v2. Instead of parsing directly to a v1 CreateTable that keeps metadata in CatalogTable, and then converting that v1 metadata to the v2 equivalent, the sql CreateTableStatement plan is produced by the parser and converted once into both implementations.

     Parsed logical plans are not resolved because they must be converted to concrete logical plans.

     Parsed logical plans are located in Catalyst so that as much SQL parsing logic as possible is be kept in a org.apache.spark.sql.catalyst.parser.AbstractSqlParser.

107.  case class Pivot(groupByExprsOpt: Option[Seq[NamedExpression]], pivotColumn: Expression, pivotValues: Seq[Expression], aggregates: Seq[Expression], child: LogicalPlan) extends UnaryNode with Product with Serializable

     A constructor for creating a pivot, which will later be converted to a Project or an Aggregate during the query analysis.

     A constructor for creating a pivot, which will later be converted to a Project or an Aggregate during the query analysis.

     groupByExprsOpt

     A sequence of group by expressions. This field should be None if coming from SQL, in which group by expressions are not explicitly specified.

     pivotColumn

     The pivot column.

     pivotValues

     A sequence of values for the pivot column.

     aggregates

     The aggregation expressions, each with or without an alias.

     child

     Child operator

108.  case class Project(projectList: Seq[NamedExpression], child: LogicalPlan) extends OrderPreservingUnaryNode with Product with Serializable
109.  case class QualifiedColType(name: Seq[String], dataType: DataType, nullable: Boolean, comment: Option[String], position: Option[ColumnPosition]) extends Product with Serializable

     Column data as parsed by ALTER TABLE ...

     Column data as parsed by ALTER TABLE ... ADD COLUMNS.

110.  trait QueryPlanConstraints extends ConstraintHelper
111.  case class Range(start: Long, end: Long, step: Long, numSlices: Option[Int], output: Seq[Attribute], isStreaming: Boolean) extends LeafNode with MultiInstanceRelation with Product with Serializable
112.  case class RefreshTable(catalog: TableCatalog, ident: Identifier) extends LogicalPlan with Command with Product with Serializable

     The logical plan of the REFRESH TABLE command that works for v2 catalogs.

113.  case class RefreshTableStatement(tableName: Seq[String]) extends ParsedStatement with Product with Serializable

     A REFRESH TABLE statement, as parsed from SQL

114.  case class RenameTable(catalog: TableCatalog, oldIdent: Identifier, newIdent: Identifier) extends LogicalPlan with Command with Product with Serializable

     The logical plan of the ALTER TABLE RENAME command that works for v2 tables.

115.  case class RenameTableStatement(oldName: Seq[String], newName: Seq[String], isView: Boolean) extends ParsedStatement with Product with Serializable

     ALTER TABLE ...

     ALTER TABLE ... RENAME TO command, as parsed from SQL.

116.  case class RepairTableStatement(tableName: Seq[String]) extends ParsedStatement with Product with Serializable

     A REPAIR TABLE statement, as parsed from SQL

117.  case class Repartition(numPartitions: Int, shuffle: Boolean, child: LogicalPlan) extends RepartitionOperation with Product with Serializable

     Returns a new RDD that has exactly numPartitions partitions.

     Returns a new RDD that has exactly numPartitions partitions. Differs from RepartitionByExpression as this method is called directly by DataFrame's, because the user asked for coalesce or repartition. RepartitionByExpression is used when the consumer of the output requires some specific ordering or distribution of the data.

118.  case class RepartitionByExpression(partitionExpressions: Seq[Expression], child: LogicalPlan, numPartitions: Int) extends RepartitionOperation with Product with Serializable

     This method repartitions data using Expressions into numPartitions, and receives information about the number of partitions during execution.

     This method repartitions data using Expressions into numPartitions, and receives information about the number of partitions during execution. Used when a specific ordering or distribution is expected by the consumer of the query result. Use Repartition for RDD-like coalesce and repartition.

119.  abstract class RepartitionOperation extends UnaryNode

     A base interface for RepartitionByExpression and Repartition

120.  case class ReplaceTable(catalog: TableCatalog, tableName: Identifier, tableSchema: StructType, partitioning: Seq[Transform], properties: Map[String, String], orCreate: Boolean) extends LogicalPlan with Command with V2CreateTablePlan with Product with Serializable

     Replace a table with a v2 catalog.

     Replace a table with a v2 catalog.

     If the table does not exist, and orCreate is true, then it will be created. If the table does not exist, and orCreate is false, then an exception will be thrown.

     The persisted table will have no contents as a result of this operation.

121.  case class ReplaceTableAsSelect(catalog: TableCatalog, tableName: Identifier, partitioning: Seq[Transform], query: LogicalPlan, properties: Map[String, String], writeOptions: Map[String, String], orCreate: Boolean) extends LogicalPlan with Command with V2CreateTablePlan with Product with Serializable

     Replaces a table from a select query with a v2 catalog.

     Replaces a table from a select query with a v2 catalog.

     If the table does not exist, and orCreate is true, then it will be created. If the table does not exist, and orCreate is false, then an exception will be thrown.

122.  case class ReplaceTableAsSelectStatement(tableName: Seq[String], asSelect: LogicalPlan, partitioning: Seq[Transform], bucketSpec: Option[BucketSpec], properties: Map[String, String], provider: Option[String], options: Map[String, String], location: Option[String], comment: Option[String], writeOptions: Map[String, String], orCreate: Boolean) extends ParsedStatement with Product with Serializable

     A REPLACE TABLE AS SELECT command, as parsed from SQL.

123.  case class ReplaceTableStatement(tableName: Seq[String], tableSchema: StructType, partitioning: Seq[Transform], bucketSpec: Option[BucketSpec], properties: Map[String, String], provider: Option[String], options: Map[String, String], location: Option[String], comment: Option[String], orCreate: Boolean) extends ParsedStatement with Product with Serializable

     A REPLACE TABLE command, as parsed from SQL.

     A REPLACE TABLE command, as parsed from SQL.

     If the table exists prior to running this command, executing this statement will replace the table's metadata and clear the underlying rows from the table.

124.  case class ResolvedHint(child: LogicalPlan, hints: HintInfo = HintInfo()) extends UnaryNode with Product with Serializable

     A resolved hint node.

     A resolved hint node. The analyzer should convert all UnresolvedHint into ResolvedHint. This node will be eliminated before optimization starts.

125.  case class ReturnAnswer(child: LogicalPlan) extends UnaryNode with Product with Serializable

     When planning take() or collect() operations, this special node that is inserted at the top of the logical plan before invoking the query planner.

     When planning take() or collect() operations, this special node that is inserted at the top of the logical plan before invoking the query planner.

     Rules can pattern-match on this node in order to apply transformations that only take effect at the top of the logical query plan.

126.  case class Sample(lowerBound: Double, upperBound: Double, withReplacement: Boolean, seed: Long, child: LogicalPlan) extends UnaryNode with Product with Serializable

     Sample the dataset.

     Sample the dataset.

     lowerBound

     Lower-bound of the sampling probability (usually 0.0)

     upperBound

     Upper-bound of the sampling probability. The expected fraction sampled will be ub - lb.

     withReplacement

     Whether to sample with replacement.

     seed

     the random seed

     child

     the LogicalPlan

127.  case class ScriptInputOutputSchema(inputRowFormat: Seq[(String, String)], outputRowFormat: Seq[(String, String)], inputSerdeClass: Option[String], outputSerdeClass: Option[String], inputSerdeProps: Seq[(String, String)], outputSerdeProps: Seq[(String, String)], recordReaderClass: Option[String], recordWriterClass: Option[String], schemaLess: Boolean) extends Product with Serializable

     Input and output properties when passing data to a script.

     Input and output properties when passing data to a script. For example, in Hive this would specify which SerDes to use.

128.  case class ScriptTransformation(input: Seq[Expression], script: String, output: Seq[Attribute], child: LogicalPlan, ioschema: ScriptInputOutputSchema) extends UnaryNode with Product with Serializable

     Transforms the input by forking and running the specified script.

     Transforms the input by forking and running the specified script.

     input

     the set of expression that should be passed to the script.

     script

     the command that should be executed.

     output

     the attributes that are produced by the script.

     ioschema

     the input and output schema applied in the execution of the script.

129.  case class SerializeFromObject(serializer: Seq[NamedExpression], child: LogicalPlan) extends UnaryNode with ObjectConsumer with Product with Serializable

     Takes the input object from child and turns it into unsafe row using the given serializer expression.

130.  case class SetCatalogAndNamespace(catalogManager: CatalogManager, catalogName: Option[String], namespace: Option[Seq[String]]) extends LogicalPlan with Command with Product with Serializable

     The logical plan of the USE/USE NAMESPACE command that works for v2 catalogs.

131.  abstract class SetOperation extends BinaryNode
132.  case class ShowColumnsStatement(table: Seq[String], namespace: Option[Seq[String]]) extends ParsedStatement with Product with Serializable

     A SHOW COLUMNS statement, as parsed from SQL

133.  case class ShowCreateTableStatement(tableName: Seq[String], asSerde: Boolean = false) extends ParsedStatement with Product with Serializable

     A SHOW CREATE TABLE statement, as parsed from SQL.

134.  case class ShowCurrentNamespace(catalogManager: CatalogManager) extends LogicalPlan with Command with Product with Serializable

     The logical plan of the SHOW CURRENT NAMESPACE command that works for v2 catalogs.

135.  case class ShowCurrentNamespaceStatement() extends ParsedStatement with Product with Serializable

     A SHOW CURRENT NAMESPACE statement, as parsed from SQL

136.  case class ShowFunctionsStatement(userScope: Boolean, systemScope: Boolean, pattern: Option[String], functionName: Option[Seq[String]]) extends ParsedStatement with Product with Serializable

     SHOW FUNCTIONS statement, as parsed from SQL

137.  case class ShowNamespaces(namespace: LogicalPlan, pattern: Option[String]) extends LogicalPlan with Command with Product with Serializable

     The logical plan of the SHOW NAMESPACES command that works for v2 catalogs.

138.  case class ShowPartitionsStatement(tableName: Seq[String], partitionSpec: Option[TablePartitionSpec]) extends ParsedStatement with Product with Serializable

     A SHOW PARTITIONS statement, as parsed from SQL

139.  case class ShowTableProperties(table: LogicalPlan, propertyKey: Option[String]) extends LogicalPlan with Command with Product with Serializable

     The logical plan of the SHOW TBLPROPERTIES command that works for v2 catalogs.

140.  case class ShowTableStatement(namespace: Option[Seq[String]], pattern: String, partitionSpec: Option[TablePartitionSpec]) extends ParsedStatement with Product with Serializable

     A SHOW TABLE EXTENDED statement, as parsed from SQL.

141.  case class ShowTables(namespace: LogicalPlan, pattern: Option[String]) extends LogicalPlan with Command with Product with Serializable

     The logical plan of the SHOW TABLE command that works for v2 catalogs.

142.  case class ShowViews(namespace: LogicalPlan, pattern: Option[String]) extends LogicalPlan with Command with Product with Serializable

     The logical plan of the SHOW VIEWS command that works for v1 and v2 catalogs.

     The logical plan of the SHOW VIEWS command that works for v1 and v2 catalogs.

     Notes: v2 catalogs do not support views API yet, the command will fallback to v1 ShowViewsCommand during ResolveSessionCatalog.

143.  case class Sort(order: Seq[SortOrder], global: Boolean, child: LogicalPlan) extends UnaryNode with Product with Serializable
     order

     The ordering expressions

     global

     True means global sorting apply for entire data set, False means sorting only apply within the partition.

     child

     Child logical plan

144.  case class Statistics(sizeInBytes: BigInt, rowCount: Option[BigInt] = None, attributeStats: AttributeMap[ColumnStat] = AttributeMap(Nil)) extends Product with Serializable

     Estimates of various statistics.

     Estimates of various statistics. The default estimation logic simply lazily multiplies the corresponding statistic produced by the children. To override this behavior, override statistics and assign it an overridden version of Statistics.

     NOTE: concrete and/or overridden versions of statistics fields should pay attention to the performance of the implementations. The reason is that estimations might get triggered in performance-critical processes, such as query plan planning.

     Note that we are using a BigInt here since it is easy to overflow a 64-bit integer in cardinality estimation (e.g. cartesian joins).

     sizeInBytes

     Physical size in bytes. For leaf operators this defaults to 1, otherwise it defaults to the product of children's sizeInBytes.

     rowCount

     Estimated number of rows.

     attributeStats

     Statistics for Attributes.

145.  case class Subquery(child: LogicalPlan, correlated: Boolean) extends OrderPreservingUnaryNode with Product with Serializable

     This node is inserted at the top of a subquery when it is optimized.

     This node is inserted at the top of a subquery when it is optimized. This makes sure we can recognize a subquery as such, and it allows us to write subquery aware transformations.

     correlated

     flag that indicates the subquery is correlated, and will be rewritten into a join during analysis.

146.  case class SubqueryAlias(identifier: AliasIdentifier, child: LogicalPlan) extends OrderPreservingUnaryNode with Product with Serializable

     Aliased subquery.

     Aliased subquery.

     identifier

     the alias identifier for this subquery.

     child

     the logical plan of this subquery.

147.  trait SupportsSubquery extends LogicalPlan

     A trait to represent the commands that support subqueries.

     A trait to represent the commands that support subqueries. This is used to whitelist such commands in the subquery-related checks.

148.  case class Tail(limitExpr: Expression, child: LogicalPlan) extends OrderPreservingUnaryNode with Product with Serializable

     This is similar with Limit except:

     This is similar with Limit except:

     - It does not have plans for global/local separately because currently there is only single implementation which initially mimics both global/local tails. See org.apache.spark.sql.execution.CollectTailExec and org.apache.spark.sql.execution.CollectLimitExec

     - Currently, this plan can only be a root node.

149.  case class TruncateTableStatement(tableName: Seq[String], partitionSpec: Option[TablePartitionSpec]) extends ParsedStatement with Product with Serializable

     A TRUNCATE TABLE statement, as parsed from SQL

150.  case class TypedFilter(func: AnyRef, argumentClass: Class[_], argumentSchema: StructType, deserializer: Expression, child: LogicalPlan) extends UnaryNode with Product with Serializable

     A relation produced by applying func to each element of the child and filter them by the resulting boolean value.

     A relation produced by applying func to each element of the child and filter them by the resulting boolean value.

     This is logically equal to a normal Filter operator whose condition expression is decoding the input row to object and apply the given function with decoded object. However we need the encapsulation of TypedFilter to make the concept more clear and make it easier to write optimizer rules.

151.  abstract class UnaryNode extends LogicalPlan

     A logical plan node with single child.

152.  case class UncacheTableStatement(tableName: Seq[String], ifExists: Boolean) extends ParsedStatement with Product with Serializable

     An UNCACHE TABLE statement, as parsed from SQL

153.  case class Union(children: Seq[LogicalPlan]) extends LogicalPlan with Product with Serializable

     Logical plan for unioning two plans, without a distinct.

     Logical plan for unioning two plans, without a distinct. This is UNION ALL in SQL.

154.  case class UnresolvedHint(name: String, parameters: Seq[Any], child: LogicalPlan) extends UnaryNode with Product with Serializable

     A general hint for the child that is not yet resolved.

     A general hint for the child that is not yet resolved. This node is generated by the parser and should be removed This node will be eliminated post analysis.

     name

     the name of the hint

     parameters

     the parameters of the hint

     child

     the LogicalPlan on which this hint applies

155.  case class UpdateAction(condition: Option[Expression], assignments: Seq[Assignment]) extends MergeAction with Product with Serializable
156.  case class UpdateTable(table: LogicalPlan, assignments: Seq[Assignment], condition: Option[Expression]) extends LogicalPlan with Command with SupportsSubquery with Product with Serializable

     The logical plan of the UPDATE TABLE command that works for v2 tables.

157.  case class UseStatement(isNamespaceSet: Boolean, nameParts: Seq[String]) extends ParsedStatement with Product with Serializable

     A USE statement, as parsed from SQL.

158.  trait V2CreateTablePlan extends LogicalPlan

     A trait used for logical plan nodes that create or replace V2 table definitions.

159.  trait V2WriteCommand extends LogicalPlan with Command

     Base trait for DataSourceV2 write commands

160.  case class View(desc: CatalogTable, output: Seq[Attribute], child: LogicalPlan) extends LogicalPlan with MultiInstanceRelation with Product with Serializable

     A container for holding the view description(CatalogTable), and the output of the view.

     A container for holding the view description(CatalogTable), and the output of the view. The child should be a logical plan parsed from the CatalogTable.viewText, should throw an error if the viewText is not defined. This operator will be removed at the end of analysis stage.

     desc

     A view description(CatalogTable) that provides necessary information to resolve the view.

     output

     The output of a view operator, this is generated during planning the view, so that we are able to decouple the output from the underlying structure.

     child

     The logical plan of a view operator, it should be a logical plan parsed from the CatalogTable.viewText, should throw an error if the viewText is not defined.

161.  case class Window(windowExpressions: Seq[NamedExpression], partitionSpec: Seq[Expression], orderSpec: Seq[SortOrder], child: LogicalPlan) extends UnaryNode with Product with Serializable
162.  case class With(child: LogicalPlan, cteRelations: Seq[(String, SubqueryAlias)]) extends UnaryNode with Product with Serializable

     A container for holding named common table expressions (CTEs) and a query plan.

     A container for holding named common table expressions (CTEs) and a query plan. This operator will be removed during analysis and the relations will be substituted into child.

     child

     The final query of this CTE.

     cteRelations

     A sequence of pair (alias, the CTE definition) that this CTE defined Each CTE can see the base tables and the previously defined CTEs only.

163.  case class WithWindowDefinition(windowDefinitions: Map[String, WindowSpecDefinition], child: LogicalPlan) extends UnaryNode with Product with Serializable