B C D E F G H I M N O P S T V W
All Classes All Packages

B

bloomFilterExpectedEntries(Long) - Method in class org.projectnessie.gc.base.ImmutableGCParams.Builder

Initializes the value for the bloomFilterExpectedEntries attribute.

bloomFilterFpp(double) - Method in class org.projectnessie.gc.base.ImmutableGCParams.Builder

Initializes the value for the bloomFilterFpp attribute.

build() - Method in class org.projectnessie.gc.base.ImmutableGCParams.Builder

Builds a new ImmutableGCParams.

builder() - Static method in class org.projectnessie.gc.base.ImmutableGCParams

Creates a builder for ImmutableGCParams.

C

collectExpiredContentsAsDataSet(String) - Method in class org.projectnessie.gc.base.IdentifiedResultsRepo

Collect the expired contents for the given run id as spark dataset.

commitProtectionDuration(Duration) - Method in class org.projectnessie.gc.base.ImmutableGCParams.Builder

Initializes the value for the commitProtectionDuration attribute.

computeLiveContentsFunc(long, Map<String, Instant>) - Method in class org.projectnessie.gc.base.IdentifyContentsPerExecutor

ContentBloomFilter - Class in org.projectnessie.gc.base

A utility class wrapping bloom filter functionality.

ContentBloomFilter(long, double) - Constructor for class org.projectnessie.gc.base.ContentBloomFilter

copyOf(GCParams) - Static method in class org.projectnessie.gc.base.ImmutableGCParams

Creates an immutable copy of a GCParams value.

cutOffTimestampPerRef(Map<String, ? extends Instant>) - Method in class org.projectnessie.gc.base.ImmutableGCParams.Builder

Sets or replaces all mappings from the specified map as entries for the cutOffTimestampPerRef map.

D

deadReferenceCutOffTimeStamp(Instant) - Method in class org.projectnessie.gc.base.ImmutableGCParams.Builder

Initializes the value for the deadReferenceCutOffTimeStamp attribute.

defaultCutOffTimestamp(Instant) - Method in class org.projectnessie.gc.base.ImmutableGCParams.Builder

Initializes the value for the defaultCutOffTimestamp attribute.

deserializeReference(String) - Static method in class org.projectnessie.gc.base.GCUtil

Deserialize JSON String to Reference object.

DistributedIdentifyContents - Class in org.projectnessie.gc.base

Identify the expired and live contents in a distributed way using the spark and bloom filter by walking all the references (both dead and live).

DistributedIdentifyContents(SparkSession, GCParams) - Constructor for class org.projectnessie.gc.base.DistributedIdentifyContents

E

equals(Object) - Method in class org.projectnessie.gc.base.ImmutableGCParams

This instance is equal to all instances of ImmutableGCParams that have equal attribute values.

F

from(GCParams) - Method in class org.projectnessie.gc.base.ImmutableGCParams.Builder

Fill a builder with attribute values from the provided GCParams instance.

G

GCImpl - Class in org.projectnessie.gc.base

Encapsulates the logic to retrieve expired contents by walking over all commits in all named-references.

GCImpl(GCParams) - Constructor for class org.projectnessie.gc.base.GCImpl

Instantiates a new GCImpl.

GCParams - Interface in org.projectnessie.gc.base

Config params for GC.

GCUtil - Class in org.projectnessie.gc.base

getBloomFilterExpectedEntries() - Method in interface org.projectnessie.gc.base.GCParams

Optional bloom filter expected live commits entries per reference.

getBloomFilterExpectedEntries() - Method in class org.projectnessie.gc.base.ImmutableGCParams

Optional bloom filter expected live commits entries per reference.

getBloomFilterFpp() - Method in interface org.projectnessie.gc.base.GCParams

Optional bloom filter fpp.

getBloomFilterFpp() - Method in class org.projectnessie.gc.base.ImmutableGCParams

Optional bloom filter fpp.

getCommitProtectionDuration() - Method in interface org.projectnessie.gc.base.GCParams

Commit protection duration to avoid expiring on going or recent commits.

getCommitProtectionDuration() - Method in class org.projectnessie.gc.base.ImmutableGCParams

Commit protection duration to avoid expiring on going or recent commits.

getCutOffTimestampPerRef() - Method in interface org.projectnessie.gc.base.GCParams

Optional cutoff time per live reference.

getCutOffTimestampPerRef() - Method in class org.projectnessie.gc.base.ImmutableGCParams

Optional cutoff time per live reference.

getDeadReferenceCutOffTimeStamp() - Method in interface org.projectnessie.gc.base.GCParams

Optional cutoff time for all the dead references.

getDeadReferenceCutOffTimeStamp() - Method in class org.projectnessie.gc.base.ImmutableGCParams

Optional cutoff time for all the dead references.

getDefaultCutOffTimestamp() - Method in interface org.projectnessie.gc.base.GCParams

Default cutoff time for all the references.

getDefaultCutOffTimestamp() - Method in class org.projectnessie.gc.base.ImmutableGCParams

Default cutoff time for all the references.

getExpectedFpp() - Method in class org.projectnessie.gc.base.ContentBloomFilter

getExpiredContentRowsFunc(Map<String, ContentBloomFilter>, String, Timestamp) - Method in class org.projectnessie.gc.base.IdentifyContentsPerExecutor

getInstantFromMicros(Long) - Static method in class org.projectnessie.gc.base.GCUtil

getLatestCompletedRunID() - Method in class org.projectnessie.gc.base.IdentifiedResultsRepo

getLiveContentsBloomFilters(List<String>, long, Map<String, Instant>) - Method in class org.projectnessie.gc.base.DistributedIdentifyContents

Compute the bloom filter per content id by walking all the live references in a distributed way using spark.

getNessieCatalogName() - Method in interface org.projectnessie.gc.base.GCParams

Nessie catalog name to be used with spark to create the output results table.

getNessieCatalogName() - Method in class org.projectnessie.gc.base.ImmutableGCParams

Nessie catalog name to be used with spark to create the output results table.

getNessieClientConfigs() - Method in interface org.projectnessie.gc.base.GCParams

Nessie client configurations from NessieConfigConstants.

getNessieClientConfigs() - Method in class org.projectnessie.gc.base.ImmutableGCParams

Nessie client configurations from NessieConfigConstants.

getOutputBranchName() - Method in interface org.projectnessie.gc.base.GCParams

Branch's name to be used for creating the output table.

getOutputBranchName() - Method in class org.projectnessie.gc.base.ImmutableGCParams

Branch's name to be used for creating the output table.

getOutputTableIdentifier() - Method in interface org.projectnessie.gc.base.GCParams

Output table identifier (namespace and table name) to be used for storing the results in GCParams.getOutputBranchName().

getOutputTableIdentifier() - Method in class org.projectnessie.gc.base.ImmutableGCParams

Output table identifier (namespace and table name) to be used for storing the results in ImmutableGCParams.getOutputBranchName().

getSchema() - Method in class org.projectnessie.gc.base.IdentifiedResultsRepo

getSparkPartitionsCount() - Method in interface org.projectnessie.gc.base.GCParams

Optional spark partitions count to be used for distributing references.

getSparkPartitionsCount() - Method in class org.projectnessie.gc.base.ImmutableGCParams

Optional spark partitions count to be used for distributing references.

H

hashCode() - Method in class org.projectnessie.gc.base.ImmutableGCParams

Computes a hash code from attributes: nessieClientConfigs, cutOffTimestampPerRef, defaultCutOffTimestamp, deadReferenceCutOffTimeStamp, sparkPartitionsCount, commitProtectionDuration, bloomFilterExpectedEntries, bloomFilterFpp, nessieCatalogName, outputBranchName, outputTableIdentifier.

I

IdentifiedResultsRepo - Class in org.projectnessie.gc.base

DDL + DML functionality for the "IdentifiedResult" table.

IdentifiedResultsRepo(SparkSession, String, String, String) - Constructor for class org.projectnessie.gc.base.IdentifiedResultsRepo

IdentifyContentsPerExecutor - Class in org.projectnessie.gc.base

Contains the methods that executes in spark executor for GCImpl.identifyExpiredContents(SparkSession).

IdentifyContentsPerExecutor(GCParams) - Constructor for class org.projectnessie.gc.base.IdentifyContentsPerExecutor

identifyExpiredContents(Map<String, ContentBloomFilter>, List<String>) - Method in class org.projectnessie.gc.base.DistributedIdentifyContents

Gets the expired contents per content id by walking all the live and dead references in a distributed way using spark and checking the contents against the live bloom filter results.

identifyExpiredContents(SparkSession) - Method in class org.projectnessie.gc.base.GCImpl

Identify the expired contents using a two-step traversal algorithm.

ImmutableGCParams - Class in org.projectnessie.gc.base

Immutable implementation of GCParams.

ImmutableGCParams.Builder - Class in org.projectnessie.gc.base

Builds instances of type ImmutableGCParams.

M

merge(ContentBloomFilter) - Method in class org.projectnessie.gc.base.ContentBloomFilter

mightContain(Content) - Method in class org.projectnessie.gc.base.ContentBloomFilter

N

nessieCatalogName(String) - Method in class org.projectnessie.gc.base.ImmutableGCParams.Builder

Initializes the value for the nessieCatalogName attribute.

nessieClientConfigs(Map<String, ? extends String>) - Method in class org.projectnessie.gc.base.ImmutableGCParams.Builder

Sets or replaces all mappings from the specified map as entries for the nessieClientConfigs map.

O

org.projectnessie.gc.base - package org.projectnessie.gc.base

outputBranchName(String) - Method in class org.projectnessie.gc.base.ImmutableGCParams.Builder

Initializes the value for the outputBranchName attribute.

outputTableIdentifier(String) - Method in class org.projectnessie.gc.base.ImmutableGCParams.Builder

Initializes the value for the outputTableIdentifier attribute.

P

put(Content) - Method in class org.projectnessie.gc.base.ContentBloomFilter

putAllCutOffTimestampPerRef(Map<String, ? extends Instant>) - Method in class org.projectnessie.gc.base.ImmutableGCParams.Builder

Put all mappings from the specified map as entries to cutOffTimestampPerRef map.

putAllNessieClientConfigs(Map<String, ? extends String>) - Method in class org.projectnessie.gc.base.ImmutableGCParams.Builder

Put all mappings from the specified map as entries to nessieClientConfigs map.

putCutOffTimestampPerRef(String, Instant) - Method in class org.projectnessie.gc.base.ImmutableGCParams.Builder

Put one entry to the cutOffTimestampPerRef map.

putCutOffTimestampPerRef(Map.Entry<String, ? extends Instant>) - Method in class org.projectnessie.gc.base.ImmutableGCParams.Builder

Put one entry to the cutOffTimestampPerRef map.

putNessieClientConfigs(String, String) - Method in class org.projectnessie.gc.base.ImmutableGCParams.Builder

Put one entry to the nessieClientConfigs map.

putNessieClientConfigs(Map.Entry<String, ? extends String>) - Method in class org.projectnessie.gc.base.ImmutableGCParams.Builder

Put one entry to the nessieClientConfigs map.

S

SerializableFunction1<T,U> - Interface in org.projectnessie.gc.base

Interface that makes scala.Function1 as Serializable.

serializeReference(Reference) - Static method in class org.projectnessie.gc.base.GCUtil

Serialize Reference object using JSON Serialization.

sparkPartitionsCount(Integer) - Method in class org.projectnessie.gc.base.ImmutableGCParams.Builder

Initializes the value for the sparkPartitionsCount attribute.

T

toString() - Method in class org.projectnessie.gc.base.ImmutableGCParams

Prints the immutable value GCParams with attribute values.

V

validate() - Method in interface org.projectnessie.gc.base.GCParams

W

wasMerged() - Method in class org.projectnessie.gc.base.ContentBloomFilter

A merged bloomfilter might indicate decreased filter quality.

withBloomFilterExpectedEntries(Long) - Method in class org.projectnessie.gc.base.ImmutableGCParams

Copy the current immutable object by setting a value for the bloomFilterExpectedEntries attribute.

withBloomFilterFpp(double) - Method in class org.projectnessie.gc.base.ImmutableGCParams

Copy the current immutable object by setting a value for the bloomFilterFpp attribute.

withCommitProtectionDuration(Duration) - Method in class org.projectnessie.gc.base.ImmutableGCParams

Copy the current immutable object by setting a value for the commitProtectionDuration attribute.

withCutOffTimestampPerRef(Map<String, ? extends Instant>) - Method in class org.projectnessie.gc.base.ImmutableGCParams

Copy the current immutable object by replacing the cutOffTimestampPerRef map with the specified map.

withDeadReferenceCutOffTimeStamp(Instant) - Method in class org.projectnessie.gc.base.ImmutableGCParams

Copy the current immutable object by setting a value for the deadReferenceCutOffTimeStamp attribute.

withDefaultCutOffTimestamp(Instant) - Method in class org.projectnessie.gc.base.ImmutableGCParams

Copy the current immutable object by setting a value for the defaultCutOffTimestamp attribute.

withNessieCatalogName(String) - Method in class org.projectnessie.gc.base.ImmutableGCParams

Copy the current immutable object by setting a value for the nessieCatalogName attribute.

withNessieClientConfigs(Map<String, ? extends String>) - Method in class org.projectnessie.gc.base.ImmutableGCParams

Copy the current immutable object by replacing the nessieClientConfigs map with the specified map.

withOutputBranchName(String) - Method in class org.projectnessie.gc.base.ImmutableGCParams

Copy the current immutable object by setting a value for the outputBranchName attribute.

withOutputTableIdentifier(String) - Method in class org.projectnessie.gc.base.ImmutableGCParams

Copy the current immutable object by setting a value for the outputTableIdentifier attribute.

withSparkPartitionsCount(Integer) - Method in class org.projectnessie.gc.base.ImmutableGCParams

Copy the current immutable object by setting a value for the sparkPartitionsCount attribute.

B C D E F G H I M N O P S T V W
All Classes All Packages