Packages

final class ColumnarBatch extends AutoCloseable

This class wraps multiple ColumnVectors as a row-wise table. It provides a row view of this batch so that Spark can access the data row by row. Instance of it is meant to be reused during the entire data loading process.

Linear Supertypes
AutoCloseable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. ColumnarBatch
  2. AutoCloseable
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new ColumnarBatch(columns: Array[ColumnVector], numRows: Int)

    Create a new batch from existing column vectors.

    Create a new batch from existing column vectors.

    columns

    The columns of this batch

    numRows

    The number of rows in this batch

  2. new ColumnarBatch(columns: Array[ColumnVector])

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. def close(): Unit

    Called to close all the columns in this batch.

    Called to close all the columns in this batch. It is not valid to access the data after calling this. This must be called at the end to clean up memory allocations.

    Definition Classes
    ColumnarBatch → AutoCloseable
  7. def column(ordinal: Int): ColumnVector

    Returns the column at ordinal.

  8. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  9. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  10. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  11. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  12. def getRow(rowId: Int): InternalRow

    Returns the row in this batch at rowId.

    Returns the row in this batch at rowId. Returned row is reused across calls.

  13. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  14. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  15. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  16. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  17. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  18. def numCols(): Int

    Returns the number of columns that make up this batch.

  19. def numRows(): Int

    Returns the number of rows for read, including filtered rows.

  20. def rowIterator(): Iterator[InternalRow]

    Returns an iterator over the rows in this batch.

  21. def setNumRows(numRows: Int): Unit

    Sets the number of rows in this batch.

  22. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  23. def toString(): String
    Definition Classes
    AnyRef → Any
  24. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  25. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  26. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from AutoCloseable

Inherited from AnyRef

Inherited from Any

Ungrouped