Spark Project Catalyst 3.0.1 API < Back

Packages

package root

Definition Classes
root
package org

Definition Classes
root
package apache

Definition Classes
org
package spark

Definition Classes
apache
package sql

Definition Classes
spark
package catalyst
Catalyst is a library for manipulating relational query plans.
Catalyst is a library for manipulating relational query plans. All classes in catalyst are considered an internal API to Spark SQL and are subject to change between minor releases.

Definition Classes
sql
package expressions
A set of classes that can be used to represent trees of relational expressions.
A set of classes that can be used to represent trees of relational expressions. A key goal of the expression library is to hide the details of naming and scoping from developers who want to manipulate trees of relational operators. As such, the library defines a special type of expression, a NamedExpression in addition to the standard collection of expressions.
Standard Expressions
A library of standard expressions (e.g., Add, EqualTo), aggregates (e.g., SUM, COUNT), and other computations (e.g. UDFs). Each expression type is capable of determining its output schema as a function of its children's output schema.
Named Expressions
Some expression are named and thus can be referenced by later operators in the dataflow graph. The two types of named expressions are AttributeReferences and Aliases. AttributeReferences refer to attributes of the input tuple for a given operator and form the leaves of some expression trees. Aliases assign a name to intermediate computations. For example, in the SQL statement SELECT a+b AS c FROM ..., the expressions a and b would be represented by AttributeReferences and c would be represented by an Alias.
During analysis, all named expressions are assigned a globally unique expression id, which can be used for equality comparisons. While the original names are kept around for debugging purposes, they should never be used to check if two attributes refer to the same value, as plan transformations can result in the introduction of naming ambiguity. For example, consider a plan that contains subqueries, both of which are reading from the same table. If an optimization removes the subqueries, scoping information would be destroyed, eliminating the ability to reason about which subquery produced a given attribute.
Evaluation
The result of expressions can be evaluated using the Expression.apply(Row) method.

Definition Classes
catalyst
package codegen
A collection of generators that build custom bytecode at runtime for performing the evaluation of catalyst expression.
A collection of generators that build custom bytecode at runtime for performing the evaluation of catalyst expression.

Definition Classes
expressions
BaseMutableProjection
BaseProjection
Block
ByteCodeStats
CodeAndComment
CodeBlock
CodeFormatter
CodeGenerator
CodegenContext
CodegenFallback
DumpByteCode
EmptyBlock
ExprCode
ExprValue
ExpressionCanonicalizer
FalseLiteral
GenerateMutableProjection
GenerateOrdering
GeneratePredicate
GenerateSafeProjection
GenerateUnsafeProjection
GenerateUnsafeRowJoiner
GeneratedClass
GlobalValue
Inline
JavaCode
LazilyGeneratedOrdering
LiteralValue
SimpleExprValue
SubExprCodes
SubExprEliminationState
TrueLiteral
UnsafeArrayWriter
UnsafeRowJoiner
UnsafeRowWriter
UnsafeWriter
VariableValue

org.apache.spark.sql.catalyst.expressions.codegen

UnsafeWriter

abstract class UnsafeWriter extends AnyRef

Base class for writing Unsafe* structures.

Linear Supertypes

AnyRef, Any

Known Subclasses

UnsafeArrayWriter, UnsafeRowWriter

Ordering

Alphabetic
By Inheritance

Inherited

UnsafeWriter
AnyRef
Any

Hide All
Show All

Visibility

Public
All

Instance Constructors

new UnsafeWriter(holder: BufferHolder)

Attributes
protected[codegen]

Abstract Value Members

abstract def setNull1Bytes(ordinal: Int): Unit
abstract def setNull2Bytes(ordinal: Int): Unit
abstract def setNull4Bytes(ordinal: Int): Unit
abstract def setNull8Bytes(ordinal: Int): Unit
abstract def write(ordinal: Int, input: Decimal, precision: Int, scale: Int): Unit
abstract def write(ordinal: Int, value: Double): Unit
abstract def write(ordinal: Int, value: Float): Unit
abstract def write(ordinal: Int, value: Long): Unit
abstract def write(ordinal: Int, value: Int): Unit
abstract def write(ordinal: Int, value: Short): Unit
abstract def write(ordinal: Int, value: Byte): Unit
abstract def write(ordinal: Int, value: Boolean): Unit

Concrete Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
def clone(): AnyRef

Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws( ... ) @native()
final def cursor(): Int
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def finalize(): Unit

Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getBuffer(): Array[Byte]
final def getBufferHolder(): BufferHolder
Accessor methods are delegated from BufferHolder class
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
Annotations
@native()
final def grow(neededSize: Int): Unit
def hashCode(): Int

Definition Classes
AnyRef → Any
Annotations
@native()
final def increaseCursor(val: Int): Unit
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
Annotations
@native()
final def notifyAll(): Unit

Definition Classes
AnyRef
Annotations
@native()
final def reset(): Unit
def setOffsetAndSize(ordinal: Int, currentCursor: Int, size: Int): Unit

Attributes
protected[codegen]
def setOffsetAndSize(ordinal: Int, size: Int): Unit

Attributes
protected[codegen]
final def setOffsetAndSizeFromPreviousCursor(ordinal: Int, previousCursor: Int): Unit
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
AnyRef → Any
final def totalSize(): Int
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... ) @native()
final def write(array: UnsafeArrayData): Unit
final def write(ordinal: Int, map: UnsafeMapData): Unit
final def write(ordinal: Int, row: UnsafeRow): Unit
final def write(ordinal: Int, input: CalendarInterval): Unit
final def write(ordinal: Int, input: Array[Byte], offset: Int, numBytes: Int): Unit
final def write(ordinal: Int, input: Array[Byte]): Unit
final def write(ordinal: Int, input: UTF8String): Unit
final def writeBoolean(offset: Long, value: Boolean): Unit

Attributes
protected[codegen]
final def writeByte(offset: Long, value: Byte): Unit

Attributes
protected[codegen]
final def writeDouble(offset: Long, value: Double): Unit

Attributes
protected[codegen]
final def writeFloat(offset: Long, value: Float): Unit

Attributes
protected[codegen]
final def writeInt(offset: Long, value: Int): Unit

Attributes
protected[codegen]
final def writeLong(offset: Long, value: Long): Unit

Attributes
protected[codegen]
final def writeShort(offset: Long, value: Short): Unit

Attributes
protected[codegen]
final def zeroOutPaddingBytes(numBytes: Int): Unit

Attributes
protected[codegen]

Packages

Standard Expressions

Named Expressions

Evaluation

UnsafeWriter

abstract class UnsafeWriter extends AnyRef

Instance Constructors

Abstract Value Members

Concrete Value Members

Inherited from AnyRef

Inherited from Any

Ungrouped

Packages

Standard Expressions

Named Expressions

Evaluation

UnsafeWriter 

abstract class UnsafeWriter extends AnyRef

Instance Constructors

Abstract Value Members

Concrete Value Members

Inherited from AnyRef

Inherited from Any

Ungrouped

UnsafeWriter