Value Members

1.  final def !=(arg0: Any): Boolean

   Definition Classes

   AnyRef → Any

2.  final def ##(): Int

   Definition Classes

   AnyRef → Any

3.  final def ==(arg0: Any): Boolean

   Definition Classes

   AnyRef → Any

4.  final def asInstanceOf[T0]: T0

   Definition Classes

   Any

5.  def assertUnresolved(): Unit

   The process of resolution to a given schema throws away information about where a given field is being bound by ordinal instead of by name.

   The process of resolution to a given schema throws away information about where a given field is being bound by ordinal instead of by name. This method checks to make sure this process has not been done already in places where we plan to do later composition of encoders.

6.  val attrs: Seq[String]

   Attributes

   protected

7.  def clone(): AnyRef

   Attributes

   protected[lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( ... ) @native()

8.  val clsTag: ClassTag[T]

   A ClassTag that can be used to construct an Array to contain a collection of T.

   A ClassTag that can be used to construct an Array to contain a collection of T.

   Definition Classes

   ExpressionEncoder → Encoder

9.  def createDeserializer(): Deserializer[T]

   Create a deserializer that can convert a Spark SQL Row into an object of type T.

   Create a deserializer that can convert a Spark SQL Row into an object of type T.

   Note that you must resolveAndBind an encoder to a specific schema before you can create a deserializer.

10.  def createSerializer(): Serializer[T]

    Create a serializer that can convert an object of type T to a Spark SQL Row.

    Create a serializer that can convert an object of type T to a Spark SQL Row.

    Note that the returned Serializer is not thread safe. Multiple calls to serializer.apply(..) are allowed to return the same actual InternalRow object. Thus, the caller should copy the result before making another call if required.

11.  val deserializer: Expression

    Returns an expression that can be used to deserialize an input row to an object of type T with a compatible schema.

    Returns an expression that can be used to deserialize an input row to an object of type T with a compatible schema. Fields of the row will be extracted using UnresolvedAttribute. of the same name as the constructor arguments.

    For complex objects that are encoded to structs, Fields of the struct will be extracted using GetColumnByOrdinal with corresponding ordinal.

12.  final def eq(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

13.  def finalize(): Unit

    Attributes

    protected[lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( classOf[java.lang.Throwable] )

14.  final def getClass(): Class[_]

    Definition Classes

    AnyRef → Any

    Annotations

    @native()

15.  final def isInstanceOf[T0]: Boolean

    Definition Classes

    Any

16.  def isSerializedAsStruct: Boolean

    Returns true if the type T is serialized as a struct by objSerializer.

17.  def isSerializedAsStructForTopLevel: Boolean

    If the type T is serialized as a struct, when it is encoded to a Spark SQL row, fields in the struct are naturally mapped to top-level columns in a row.

    If the type T is serialized as a struct, when it is encoded to a Spark SQL row, fields in the struct are naturally mapped to top-level columns in a row. In other words, the serialized struct is flattened to row. But in case of the T is also an Option type, it can't be flattened to top-level row, because in Spark SQL top-level row can't be null. This method returns true if T is serialized as struct and is not Option type.

18.  def namedExpressions: Seq[NamedExpression]

    Returns a new set (with unique ids) of NamedExpression that represent the serialized form of this object.

19.  final def ne(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

20.  final def notify(): Unit

    Definition Classes

    AnyRef

    Annotations

    @native()

21.  final def notifyAll(): Unit

    Definition Classes

    AnyRef

    Annotations

    @native()

22.  val objDeserializer: Expression
23.  val objSerializer: Expression
24.  def resolveAndBind(attrs: Seq[Attribute] = schema.toAttributes, analyzer: Analyzer = SimpleAnalyzer): ExpressionEncoder[T]

    Returns a new copy of this encoder, where the deserializer is resolved and bound to the given schema.

    Returns a new copy of this encoder, where the deserializer is resolved and bound to the given schema.

    Note that, ideally encoder is used as a container of serde expressions, the resolution and binding stuff should happen inside query framework. However, in some cases we need to use encoder as a function to do serialization directly(e.g. Dataset.collect), then we can use this method to do resolution and binding outside of query framework.

25.  val schema: StructType

    Returns the schema of encoding this type of object as a Row.

    Returns the schema of encoding this type of object as a Row.

    Definition Classes

    ExpressionEncoder → Encoder

26.  val schemaString: String

    Attributes

    protected

27.  val serializer: Seq[NamedExpression]

    A sequence of expressions, one for each top-level field that can be used to extract the values from a raw object into an InternalRow: 1.

    A sequence of expressions, one for each top-level field that can be used to extract the values from a raw object into an InternalRow: 1. If serializer encodes a raw object to a struct, strip the outer If-IsNull and get the CreateNamedStruct. 2. For other cases, wrap the single serializer with CreateNamedStruct.

28.  final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes

    AnyRef

29.  def toString(): String

    Definition Classes

    ExpressionEncoder → AnyRef → Any

30.  final def wait(): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

31.  final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

32.  final def wait(arg0: Long): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... ) @native()