Packages

  • package root
    Definition Classes
    root
  • package org
    Definition Classes
    root
  • package apache
    Definition Classes
    org
  • package spark
    Definition Classes
    apache
  • package sql
    Definition Classes
    spark
  • package catalyst

    Catalyst is a library for manipulating relational query plans.

    Catalyst is a library for manipulating relational query plans. All classes in catalyst are considered an internal API to Spark SQL and are subject to change between minor releases.

    Definition Classes
    sql
  • package trees

    A library for easily manipulating trees of operators.

    A library for easily manipulating trees of operators. Operators that extend TreeNode are granted the following interface:

    • Scala collection like methods (foreach, map, flatMap, collect, etc)

    - transform - accepts a partial function that is used to generate a new tree. When the partial function can be applied to a given tree segment, that segment is replaced with the result. After attempting to apply the partial function to a given node, the transform function recursively attempts to apply the function to that node's children.

    • debugging support - pretty printing, easy splicing of trees, etc.
    Definition Classes
    catalyst
  • CurrentOrigin
  • Origin
  • TreeNode
  • TreeNodeRef
  • TreeNodeTag

abstract class TreeNode[BaseType <: TreeNode[BaseType]] extends Product

Self Type
BaseType
Linear Supertypes
Product, Equals, AnyRef, Any
Known Subclasses
GetColumnByOrdinal, MultiAlias, NamedRelation, ResolvedNamespace, ResolvedStar, ResolvedTable, ResolvedView, Star, UnresolvedAlias, UnresolvedAttribute, UnresolvedDeserializer, UnresolvedExtractValue, UnresolvedFunction, UnresolvedGenerator, UnresolvedHaving, UnresolvedInlineTable, UnresolvedNamespace, UnresolvedOrdinal, UnresolvedRegex, UnresolvedRelation, UnresolvedStar, UnresolvedSubqueryColumnAliases, UnresolvedTable, UnresolvedTableOrView, UnresolvedTableValuedFunction, UnresolvedV2Relation, HiveTableRelation, UnresolvedCatalogRelation, DummyExpressionHolder, Abs, Acos, Acosh, Add, AddMonths, AggregateWindowFunction, Alias, And, AnsiCast, ArrayAggregate, ArrayBasedSimpleHigherOrderFunction, ArrayBinaryLike, ArrayContains, ArrayDistinct, ArrayExcept, ArrayExists, ArrayFilter, ArrayForAll, ArrayIntersect, ArrayJoin, ArrayMax, ArrayMin, ArrayPosition, ArrayRemove, ArrayRepeat, ArraySort, ArraySortLike, ArrayTransform, ArrayUnion, ArraysOverlap, ArraysZip, Ascii, Asin, Asinh, AssertTrue, AtLeastNNonNulls, Atan, Atan2, Atanh, Attribute, AttributeReference, BRound, Base64, Bin, BinaryArithmetic, BinaryArrayExpressionWithImplicitCast, BinaryComparison, BinaryExpression, BinaryMathExpression, BinaryOperator, BitLength, BitwiseAnd, BitwiseCount, BitwiseNot, BitwiseOr, BitwiseXor, BoundReference, Bucket, CallMethodViaReflection, CaseWhen, Cast, CastBase, Cbrt, Ceil, CheckOverflow, Chr, Coalesce, CollectionGenerator, ComplexTypeMergingExpression, Concat, ConcatWs, Contains, Conv, Cos, Cosh, Cot, Crc32, CreateArray, CreateMap, CreateNamedStruct, CsvToStructs, Cube, CumeDist, CurrentBatchTimestamp, CurrentDatabase, CurrentDate, CurrentRow, CurrentTimestamp, CurrentTimestampLike, DateAdd, DateAddInterval, DateDiff, DateFormatClass, DatePart, DateSub, DayOfMonth, DayOfWeek, DayOfYear, DayWeek, Days, Decode, DenseRank, DivModLike, Divide, DivideInterval, DynamicPruning, DynamicPruningExpression, DynamicPruningSubquery, ElementAt, Elt, Encode, EndsWith, EqualNullSafe, EqualTo, EulerNumber, Exists, Exp, ExpectsInputTypes, Explode, ExplodeBase, Expm1, Expression, Extract, ExtractIntervalDays, ExtractIntervalHours, ExtractIntervalMinutes, ExtractIntervalMonths, ExtractIntervalPart, ExtractIntervalSeconds, ExtractIntervalYears, ExtractValue, Factorial, FindInSet, Flatten, Floor, FormatNumber, FormatString, FromUTCTimestamp, FromUnixTime, Generator, GeneratorOuter, GetArrayItem, GetArrayStructFields, GetJsonObject, GetMapValue, GetMapValueUtil, GetStructField, GreaterThan, GreaterThanOrEqual, Greatest, Grouping, GroupingID, GroupingSet, HashExpression, Hex, HigherOrderFunction, HiveHash, Hour, Hours, Hypot, If, IfNull, ImplicitCastInputTypes, In, InSet, InSubquery, InitCap, Inline, InputFileBlockLength, InputFileBlockStart, InputFileName, IntegralDivide, IntervalNumOperation, IsNaN, IsNotNull, IsNull, JsonToStructs, JsonTuple, KnownFloatingPointNormalized, KnownNotNull, Lag, LambdaFunction, LastDay, Lead, LeafExpression, LeafMathExpression, Least, Left, Length, LessThan, LessThanOrEqual, Levenshtein, Like, ListQuery, Literal, Log, Log10, Log1p, Log2, Logarithm, Lower, MakeDate, MakeDecimal, MakeInterval, MakeTimestamp, MapBasedSimpleHigherOrderFunction, MapConcat, MapEntries, MapFilter, MapFromArrays, MapFromEntries, MapKeys, MapValues, MapZipWith, Md5, Minute, MonotonicallyIncreasingID, Month, Months, MonthsBetween, Multiply, MultiplyInterval, Murmur3Hash, NTile, NaNvl, NamePlaceholder, NamedExpression, NamedLambdaVariable, NextDay, NonSQLExpression, Nondeterministic, Not, Now, NullIf, NullIntolerant, Nvl, Nvl2, OctetLength, OffsetWindowFunction, Or, OuterReference, Overlay, ParseToDate, ParseToTimestamp, ParseUrl, PartitionTransformExpression, PercentRank, Pi, PlanExpression, Pmod, PosExplode, Pow, PreciseTimestampConversion, Predicate, PrettyAttribute, PrintToStderr, PromotePrecision, PythonUDF, Quarter, QuaternaryExpression, RDG, RLike, Rand, Randn, Rank, RankLike, RegExpExtract, RegExpReplace, Remainder, ReplicateRows, Reverse, Right, Rint, Rollup, Round, RoundBase, RowNumber, RowNumberLike, RuntimeReplaceable, ScalaUDF, ScalarSubquery, SchemaOfCsv, SchemaOfJson, Second, SecondWithFraction, Sentences, SeptenaryExpression, Sequence, Sha1, Sha2, ShiftLeft, ShiftRight, ShiftRightUnsigned, Shuffle, Signum, SimpleHigherOrderFunction, Sin, Sinh, Size, SizeBasedWindowFunction, Slice, SortArray, SortOrder, SortPrefix, SoundEx, SparkPartitionID, SparkVersion, SpecialFrameBoundary, SpecifiedWindowFrame, Sqrt, Stack, StartsWith, Stateful, String2StringExpression, String2TrimExpression, StringInstr, StringLPad, StringLocate, StringPredicate, StringRPad, StringRegexExpression, StringRepeat, StringReplace, StringSpace, StringSplit, StringToMap, StringTranslate, StringTrim, StringTrimLeft, StringTrimRight, StructsToCsv, StructsToJson, SubqueryExpression, Substring, SubstringIndex, Subtract, SubtractDates, SubtractTimestamps, TaggingExpression, Tan, Tanh, TernaryExpression, TimeAdd, TimeSub, TimeWindow, TimeZoneAwareExpression, ToDegrees, ToRadians, ToTimestamp, ToUTCTimestamp, ToUnixTimestamp, TransformKeys, TransformValues, TruncDate, TruncInstant, TruncTimestamp, TypeOf, UnBase64, UnaryExpression, UnaryLogExpression, UnaryMathExpression, UnaryMinus, UnaryPositive, UnboundedFollowing, UnboundedPreceding, Unevaluable, UnevaluableAggregate, Unhex, UnixTime, UnixTimestamp, UnresolvedNamedLambdaVariable, UnresolvedWindowExpression, UnscaledValue, UnspecifiedFrame, UpCast, Upper, UserDefinedGenerator, Uuid, WeekDay, WeekOfYear, WindowExpression, WindowFrame, WindowFunction, WindowSpecDefinition, XxHash64, Year, YearOfWeek, Years, ZipWith, AggregateExpression, AggregateFunction, ApproxCountDistinctForIntervals, ApproximatePercentile, Average, BitAggregate, BitAndAgg, BitOrAgg, BitXorAgg, BoolAnd, BoolOr, CentralMomentAgg, Collect, CollectList, CollectSet, Corr, Count, CountIf, CountMinSketchAgg, CovPopulation, CovSample, Covariance, DeclarativeAggregate, First, HyperLogLogPlusPlus, ImperativeAggregate, Kurtosis, Last, Max, MaxBy, MaxMinBy, Min, MinBy, NoOp, PearsonCorrelation, Percentile, PivotFirst, Skewness, StddevPop, StddevSamp, Sum, TypedImperativeAggregate, UnevaluableBooleanAggBase, VariancePop, VarianceSamp, Block, CodeBlock, CodegenFallback, EmptyBlock, AssertNotNull, CatalystToExternalMap, CreateExternalRow, DecodeUsingSerializer, EncodeUsingSerializer, ExternalMapToCatalyst, GetExternalRowField, InitializeJavaBean, Invoke, InvokeLike, LambdaVariable, MapObjects, NewInstance, StaticInvoke, UnresolvedCatalystToExternalMap, UnresolvedMapObjects, UnwrapOption, ValidateExternalType, WrapOption, XPathBoolean, XPathDouble, XPathExtract, XPathFloat, XPathInt, XPathList, XPathLong, XPathShort, XPathString, NormalizeNaNAndZero, OrderedJoin, QueryPlan, Aggregate, AlterNamespaceSetLocation, AlterNamespaceSetProperties, AlterTable, AlterTableAddColumnsStatement, AlterTableAddPartitionStatement, AlterTableAlterColumnStatement, AlterTableDropColumnsStatement, AlterTableDropPartitionStatement, AlterTableRecoverPartitionsStatement, AlterTableRenameColumnStatement, AlterTableRenamePartitionStatement, AlterTableSerDePropertiesStatement, AlterTableSetLocationStatement, AlterTableSetPropertiesStatement, AlterTableUnsetPropertiesStatement, AlterViewAsStatement, AlterViewSetPropertiesStatement, AlterViewUnsetPropertiesStatement, AnalysisHelper, AnalyzeColumnStatement, AnalyzeTableStatement, AppendColumns, AppendColumnsWithObject, AppendData, ArrowEvalPython, Assignment, BaseEvalPython, BatchEvalPython, BinaryNode, CacheTableStatement, CoGroup, CollectMetrics, Command, CommentOnNamespace, CommentOnTable, CreateFunctionStatement, CreateNamespace, CreateNamespaceStatement, CreateTableAsSelect, CreateTableAsSelectStatement, CreateTableStatement, CreateV2Table, CreateViewStatement, Deduplicate, DeleteAction, DeleteFromTable, DescribeColumnStatement, DescribeFunctionStatement, DescribeNamespace, DescribeRelation, DeserializeToObject, Distinct, DropFunctionStatement, DropNamespace, DropTable, DropTableStatement, DropViewStatement, EventTimeWatermark, Except, Expand, Filter, FlatMapCoGroupsInPandas, FlatMapGroupsInPandas, FlatMapGroupsInR, FlatMapGroupsInRWithArrow, FlatMapGroupsWithState, Generate, GlobalLimit, GroupingSets, IgnoreCachedData, InsertAction, InsertIntoDir, InsertIntoStatement, Intersect, Join, LeafNode, LoadDataStatement, LocalLimit, LocalRelation, LogicalPlan, MapElements, MapGroups, MapInPandas, MapPartitions, MapPartitionsInR, MapPartitionsInRWithArrow, MergeAction, MergeIntoTable, ObjectConsumer, ObjectProducer, OneRowRelation, OrderPreservingUnaryNode, OverwriteByExpression, OverwritePartitionsDynamic, ParsedStatement, Pivot, Project, Range, RefreshTable, RefreshTableStatement, RenameTable, RenameTableStatement, RepairTableStatement, Repartition, RepartitionByExpression, RepartitionOperation, ReplaceTable, ReplaceTableAsSelect, ReplaceTableAsSelectStatement, ReplaceTableStatement, ResolvedHint, ReturnAnswer, Sample, ScriptTransformation, SerializeFromObject, SetCatalogAndNamespace, SetOperation, ShowColumnsStatement, ShowCreateTableStatement, ShowCurrentNamespace, ShowCurrentNamespaceStatement, ShowFunctionsStatement, ShowNamespaces, ShowPartitionsStatement, ShowTableProperties, ShowTableStatement, ShowTables, ShowViews, Sort, Subquery, SubqueryAlias, SupportsSubquery, Tail, TruncateTableStatement, TypedFilter, UnaryNode, UncacheTableStatement, Union, UnresolvedHint, UpdateAction, UpdateTable, UseStatement, V2CreateTablePlan, V2WriteCommand, View, Window, With, WithWindowDefinition, HashPartitioning, PartitioningCollection, RangePartitioning, DataSourceV2Relation, DataSourceV2ScanRelation, StreamingDataSourceV2Relation
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. TreeNode
  2. Product
  3. Equals
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new TreeNode()

Abstract Value Members

  1. abstract def canEqual(that: Any): Boolean
    Definition Classes
    Equals
  2. abstract def children: Seq[BaseType]

    Returns a Seq of the children of this node.

    Returns a Seq of the children of this node. Children should not change. Immutability required for containsChild optimization

  3. abstract def productArity: Int
    Definition Classes
    Product
  4. abstract def productElement(n: Int): Any
    Definition Classes
    Product
  5. abstract def simpleStringWithNodeId(): String

    ONE line description of this node containing the node identifier.

  6. abstract def verboseString(maxFields: Int): String

    ONE line description of this node with more information

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. def apply(number: Int): TreeNode[_]

    Returns the tree node at the specified number, used primarily for interactive debugging.

    Returns the tree node at the specified number, used primarily for interactive debugging. Numbers for each node can be found in the numberedTreeString.

    Note that this cannot return BaseType because logical plan's plan node might return physical plan for innerChildren, e.g. in-memory relation logical plan node has a reference to the physical plan node it is referencing.

  5. def argString(maxFields: Int): String

    Returns a string representing the arguments to this node, minus any children

  6. def asCode: String

    Returns a 'scala code' representation of this TreeNode and its children.

    Returns a 'scala code' representation of this TreeNode and its children. Intended for use when debugging where the prettier toString function is obfuscating the actual structure. In the case of 'pure' TreeNodes that only contain primitives and other TreeNodes, the result can be pasted in the REPL to build an equivalent Tree.

  7. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  8. def clone(): BaseType
    Definition Classes
    TreeNode → AnyRef
  9. def collect[B](pf: PartialFunction[BaseType, B]): Seq[B]

    Returns a Seq containing the result of applying a partial function to all elements in this tree on which the function is defined.

  10. def collectFirst[B](pf: PartialFunction[BaseType, B]): Option[B]

    Finds and returns the first TreeNode of the tree for which the given partial function is defined (pre-order), and applies the partial function to it.

  11. def collectLeaves(): Seq[BaseType]

    Returns a Seq containing the leaves in this tree.

  12. lazy val containsChild: Set[TreeNode[_]]
  13. def copyTagsFrom(other: BaseType): Unit
    Attributes
    protected
  14. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  15. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  16. def fastEquals(other: TreeNode[_]): Boolean

    Faster version of equality which short-circuits when two treeNodes are the same instance.

    Faster version of equality which short-circuits when two treeNodes are the same instance. We don't just override Object.equals, as doing so prevents the scala compiler from generating case class equals methods

  17. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  18. def find(f: (BaseType) ⇒ Boolean): Option[BaseType]

    Find the first TreeNode that satisfies the condition specified by f.

    Find the first TreeNode that satisfies the condition specified by f. The condition is recursively applied to this node and all of its children (pre-order).

  19. def flatMap[A](f: (BaseType) ⇒ TraversableOnce[A]): Seq[A]

    Returns a Seq by applying a function to all nodes in this tree and using the elements of the resulting collections.

  20. def foreach(f: (BaseType) ⇒ Unit): Unit

    Runs the given function on this node and then recursively on children.

    Runs the given function on this node and then recursively on children.

    f

    the function to be applied to each node in the tree.

  21. def foreachUp(f: (BaseType) ⇒ Unit): Unit

    Runs the given function recursively on children then on this node.

    Runs the given function recursively on children then on this node.

    f

    the function to be applied to each node in the tree.

  22. def generateTreeString(depth: Int, lastChildren: Seq[Boolean], append: (String) ⇒ Unit, verbose: Boolean, prefix: String = "", addSuffix: Boolean = false, maxFields: Int, printNodeId: Boolean): Unit

    Appends the string representation of this node and its children to the given Writer.

    Appends the string representation of this node and its children to the given Writer.

    The i-th element in lastChildren indicates whether the ancestor of the current node at depth i + 1 is the last child of its own parent node. The depth of the root node is 0, and lastChildren for the root node should be empty.

    Note that this traversal (numbering) order must be the same as getNodeNumbered.

  23. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  24. def getTagValue[T](tag: TreeNodeTag[T]): Option[T]
  25. def hashCode(): Int
    Definition Classes
    TreeNode → AnyRef → Any
  26. def innerChildren: Seq[TreeNode[_]]

    All the nodes that should be shown as a inner nested tree of this node.

    All the nodes that should be shown as a inner nested tree of this node. For example, this can be used to show sub-queries.

  27. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  28. def jsonFields: List[JField]
    Attributes
    protected
  29. def makeCopy(newArgs: Array[AnyRef]): BaseType

    Creates a copy of this type of tree node after a transformation.

    Creates a copy of this type of tree node after a transformation. Must be overridden by child classes that have constructor arguments that are not present in the productIterator.

    newArgs

    the new product arguments.

  30. def map[A](f: (BaseType) ⇒ A): Seq[A]

    Returns a Seq containing the result of applying the given function to each node in this tree in a preorder traversal.

    Returns a Seq containing the result of applying the given function to each node in this tree in a preorder traversal.

    f

    the function to be applied.

  31. def mapChildren(f: (BaseType) ⇒ BaseType): BaseType

    Returns a copy of this node where f has been applied to all the nodes in children.

  32. def mapProductIterator[B](f: (Any) ⇒ B)(implicit arg0: ClassTag[B]): Array[B]

    Efficient alternative to productIterator.map(f).toArray.

    Efficient alternative to productIterator.map(f).toArray.

    Attributes
    protected
  33. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  34. def nodeName: String

    Returns the name of this type of TreeNode.

    Returns the name of this type of TreeNode. Defaults to the class name. Note that we remove the "Exec" suffix for physical operators here.

  35. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  36. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  37. def numberedTreeString: String

    Returns a string representation of the nodes in this tree, where each operator is numbered.

    Returns a string representation of the nodes in this tree, where each operator is numbered. The numbers can be used with TreeNode.apply to easily access specific subtrees.

    The numbers are based on depth-first traversal of the tree (with innerChildren traversed first before children).

  38. val origin: Origin
  39. def otherCopyArgs: Seq[AnyRef]

    Args to the constructor that should be copied, but not transformed.

    Args to the constructor that should be copied, but not transformed. These are appended to the transformed args automatically by makeCopy

    Attributes
    protected
  40. def p(number: Int): BaseType

    Returns the tree node at the specified number, used primarily for interactive debugging.

    Returns the tree node at the specified number, used primarily for interactive debugging. Numbers for each node can be found in the numberedTreeString.

    This is a variant of apply that returns the node as BaseType (if the type matches).

  41. def prettyJson: String
  42. def productIterator: Iterator[Any]
    Definition Classes
    Product
  43. def productPrefix: String
    Definition Classes
    Product
  44. def setTagValue[T](tag: TreeNodeTag[T], value: T): Unit
  45. def simpleString(maxFields: Int): String

    ONE line description of this node.

    ONE line description of this node.

    maxFields

    Maximum number of fields that will be converted to strings. Any elements beyond the limit will be dropped.

  46. def stringArgs: Iterator[Any]

    The arguments that should be included in the arg string.

    The arguments that should be included in the arg string. Defaults to the productIterator.

    Attributes
    protected
  47. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  48. def toJSON: String
  49. def toString(): String
    Definition Classes
    TreeNode → AnyRef → Any
  50. def transform(rule: PartialFunction[BaseType, BaseType]): BaseType

    Returns a copy of this node where rule has been recursively applied to the tree.

    Returns a copy of this node where rule has been recursively applied to the tree. When rule does not apply to a given node it is left unchanged. Users should not expect a specific directionality. If a specific directionality is needed, transformDown or transformUp should be used.

    rule

    the function use to transform this nodes children

  51. def transformDown(rule: PartialFunction[BaseType, BaseType]): BaseType

    Returns a copy of this node where rule has been recursively applied to it and all of its children (pre-order).

    Returns a copy of this node where rule has been recursively applied to it and all of its children (pre-order). When rule does not apply to a given node it is left unchanged.

    rule

    the function used to transform this nodes children

  52. def transformUp(rule: PartialFunction[BaseType, BaseType]): BaseType

    Returns a copy of this node where rule has been recursively applied first to all of its children and then itself (post-order).

    Returns a copy of this node where rule has been recursively applied first to all of its children and then itself (post-order). When rule does not apply to a given node, it is left unchanged.

    rule

    the function use to transform this nodes children

  53. def treeString(append: (String) ⇒ Unit, verbose: Boolean, addSuffix: Boolean, maxFields: Int, printOperatorId: Boolean): Unit
  54. final def treeString(verbose: Boolean, addSuffix: Boolean = false, maxFields: Int = SQLConf.get.maxToStringFields, printOperatorId: Boolean = false): String
  55. final def treeString: String

    Returns a string representation of the nodes in this tree

  56. def unsetTagValue[T](tag: TreeNodeTag[T]): Unit
  57. def verboseStringWithSuffix(maxFields: Int): String

    ONE line description of this node with some suffix information

  58. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  59. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  60. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  61. def withNewChildren(newChildren: Seq[BaseType]): BaseType

    Returns a copy of this node with the children replaced.

    Returns a copy of this node with the children replaced. TODO: Validate somewhere (in debug mode?) that children are ordered correctly.

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped