Package org.projectnessie.versioned.gc

Class BinaryBloomFilter

java.lang.Object

org.projectnessie.versioned.gc.BinaryBloomFilter

All Implemented Interfaces:

Externalizable, Serializable

public class BinaryBloomFilter
extends Object
implements Externalizable

A utility class wrapping binary bloom filter functionality for spark.

Has two different serialization formats:

1. format used when exposing a result to a row (direct use of Guava's BloomFilter.readfrom operations)
2. internal format used when doing aggregation (based on Externalizable)

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor	Description
BinaryBloomFilter()

Method Summary

Modifier and Type	Method	Description
static BinaryBloomFilter	aggregate(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> data, String column)	Run aggregation on dataset to add all rows to the bloom filter.
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	filter(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> data, String column)
boolean	mightContain(byte[] bytes)
boolean	mightContain(com.google.protobuf.ByteString bytes)
boolean	mightContain(ByteBuffer bytes)
void	readExternal(ObjectInput in)
void	writeExternal(ObjectOutput out)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BinaryBloomFilter

public BinaryBloomFilter()

Method Detail

mightContain

public boolean mightContain(byte[] bytes)

mightContain

public boolean mightContain(com.google.protobuf.ByteString bytes)

mightContain

public boolean mightContain(ByteBuffer bytes)

aggregate

public static BinaryBloomFilter aggregate(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> data,
                                          String column)

Run aggregation on dataset to add all rows to the bloom filter.

Parameters:

data - a Spark Dataset to add to binary bloom filter

column - the column to aggregate on

filter

public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> filter(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> data,
                                                                     String column)

writeExternal

public void writeExternal(ObjectOutput out)
                   throws IOException

Specified by:

writeExternal in interface Externalizable

Throws:

IOException

readExternal

public void readExternal(ObjectInput in)
                  throws IOException,
                         ClassNotFoundException

Specified by:

readExternal in interface Externalizable

Throws:

IOException

ClassNotFoundException