parquet_reimpl

Type Members

1. class AvroIndexedParquetRDD[T <: IndexedRecord] extends RDD[T]

   AvroIndexedParquetRDD is an RDD which reads an index file, and (for a given CombinedFilter, which contains a predicate on the entries of the index) has as many partitions as there are unique row groups within the indexed Parquet files whose index entries satisfy the predicate.

2. class AvroParquetRDD[T <: IndexedRecord] extends RDD[T]

3. abstract class ParquetAggregateType extends ParquetType

4. case class ParquetGroupType(groupName: String, groupRepetition: Repetition.Value, groupFields: Seq[ParquetType]) extends ParquetAggregateType with Product with Serializable

5. class ParquetLister[T <: IndexedRecord] extends Logging

   ParquetLister materializes the records within a Parquet row group as an Iterator[T].

6. class ParquetPartition extends Partition

   ParquetPartition is a Partition implementation, used in our RDD implementations, which corresponds to a single row group in a single Parquet file.

7. case class ParquetPrimitiveType(name: String, repetition: Repetition.Value, length: Option[Int], primitiveType: PrimitiveType.Value, originalType: ParquetOriginalType.Value) extends ParquetType with Product with Serializable

8. case class ParquetSchemaType(schemaName: String, schemaFields: Seq[ParquetType]) extends ParquetAggregateType with Product with Serializable

9. trait ParquetType extends Serializable

   This is a reimplementation of the Type / GroupType / PrimitiveType / MessageType hierarchy from parquet.

10. class PartitionPageReadStore extends PageReadStore

    PartitionPageReadStore is a holder for mapping between the ParquetColumnDescriptor values and the PageReader values.

11. case class TypePath(head: String, suffix: Option[TypePath]) extends Product with Serializable