Type Members

1.  case class DataSourceV2Relation(table: Table, output: Seq[AttributeReference], catalog: Option[CatalogPlugin], identifier: Option[Identifier], options: CaseInsensitiveStringMap) extends LeafNode with MultiInstanceRelation with NamedRelation with Product with Serializable

   A logical plan representing a data source v2 table.

   A logical plan representing a data source v2 table.

   table

   The table that this relation represents.

   output

   the output attributes of this relation.

   catalog

   catalogPlugin for the table. None if no catalog is specified.

   identifier

   the identifier for the table. None if no identifier is defined.

   options

   The options for this table operation. It's used to create fresh ScanBuilder and WriteBuilder.

2.  case class DataSourceV2ScanRelation(table: Table, scan: Scan, output: Seq[AttributeReference]) extends LeafNode with NamedRelation with Product with Serializable

   A logical plan for a DSv2 table with a scan already created.

   A logical plan for a DSv2 table with a scan already created.

   This is used in the optimizer to push filters and projection down before conversion to physical plan. This ensures that the stats that are used by the optimizer account for the filters and projection that will be pushed down.

   table

   a DSv2 Table

   scan

   a DSv2 Scan

   output

   the output attributes of this relation

3.  case class StreamingDataSourceV2Relation(output: Seq[Attribute], scan: Scan, stream: SparkDataStream, startOffset: Option[Offset] = None, endOffset: Option[Offset] = None) extends LeafNode with MultiInstanceRelation with Product with Serializable

   A specialization of DataSourceV2Relation with the streaming bit set to true.

   A specialization of DataSourceV2Relation with the streaming bit set to true.

   Note that, this plan has a mutable reader, so Spark won't apply operator push-down for this plan, to avoid making the plan mutable. We should consolidate this plan and DataSourceV2Relation after we figure out how to apply operator push-down for streaming data sources.