org.wikidata.query.rdf.tool.rdf

Class RdfRepository

java.lang.Object

org.wikidata.query.rdf.tool.rdf.RdfRepository

public class RdfRepository
extends Object

Wrapper for communicating with the RDF repository.

Field Summary

Fields

Modifier and Type	Field and Description
protected RdfClient	rdfClient

Constructor Summary

Constructors

Constructor and Description
RdfRepository(UrisScheme uris, RdfClient rdfClient, long maxPostSize)

Method Summary

Modifier and Type	Method and Description
Instant	fetchLeftOffTime() Fetch where we left off updating the repository.
com.google.common.collect.ImmutableSetMultimap<String,String>	getRefs(Collection<String> entityIds) Get list of reference subjects connected to entity.
com.google.common.collect.ImmutableSetMultimap<String,String>	getValues(Collection<String> entityIds) Get list of value subjects connected to entity.
boolean	hasRevision(String entityId, long revision) Does the triple store have this revision or better.
Set<String>	hasRevisions(Collection<Change> candidates) Filter set of changes and see which of them really need to be updated.
int	sync(String entityId, Collection<org.openrdf.model.Statement> statements) Synchronizes the RDF repository's representation.
int	sync(String entityId, Collection<org.openrdf.model.Statement> statements, Collection<String> valueList) Synchronizes the RDF repository's representation of an entity to be exactly the provided statements.
int	syncFromChanges(Collection<Change> changes, boolean verifyResult) Sync repository from changes list.
void	updateLeftOffTime(Instant leftOffTime) Update where we left off so when fetchLeftOffTime is next called it returns leftOffTime so we can continue from there after the updater is restarted.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

rdfClient

protected final RdfClient rdfClient

Constructor Detail

RdfRepository

public RdfRepository(UrisScheme uris,
                     RdfClient rdfClient,
                     long maxPostSize)

Parameters:

maxPostSize - Max POST form content size. Should be in sync with Jetty org.eclipse.jetty.server.Request.maxFormContentSize setting. Production default is 200M, see runBlazegraph.sh file. If that setting is changed, this one should change too, otherwise we get POST errors on big updates. See: https://phabricator.wikimedia.org/T210235

Method Detail

getValues

public com.google.common.collect.ImmutableSetMultimap<String,String> getValues(Collection<String> entityIds)

Get list of value subjects connected to entity. The connection is either via statement or via reference or via qualifier.

Returns:

Set of value subjects

getRefs

public com.google.common.collect.ImmutableSetMultimap<String,String> getRefs(Collection<String> entityIds)

Get list of reference subjects connected to entity.

Returns:

Set of references

syncFromChanges

public int syncFromChanges(Collection<Change> changes,
                           boolean verifyResult)

Sync repository from changes list. Synchronizes the RDF repository's representation of an entity to be exactly the provided statements. You can think of the RDF managed for an entity as a tree rooted at the entity. The managed tree ends where the next entity's managed tree starts. For example Q23 from wikidata includes all statements about George Washington but not those about Martha (Q191789) even though she is linked by the spouse attribute. On the other hand the qualifiers on statements about George are included in George.

Parameters:

changes - List of changes.

Returns:

Number of triples modified.

sync

public int sync(String entityId,
                Collection<org.openrdf.model.Statement> statements,
                Collection<String> valueList)

Synchronizes the RDF repository's representation of an entity to be exactly the provided statements. You can think of the RDF managed for an entity as a tree rooted at the entity. The managed tree ends where the next entity's managed tree starts. For example Q23 from wikidata includes all statements about George Washington but not those about Martha (Q191789) even though she is linked by the spouse attribute. On the other hand the qualifiers on statements about George are included in George. This method is not used for actual updates but is used for tests. TODO: switch tests to use same method as actual updates do

Parameters:

entityId - id of the entity to sync

statements - all known statements about the entity

valueList - list of used values, for cleanup

Returns:

the number of statements modified

sync

public int sync(String entityId,
                Collection<org.openrdf.model.Statement> statements)

Synchronizes the RDF repository's representation. This method is not used for actual updates but is used for tests. TODO: switch tests to use same method as actual updates do

Parameters:

entityId - id of the entity to sync

statements - all known statements about the entity

Returns:

the number of statements modified

See Also:

sync(String, Collection, Collection)

hasRevisions

public Set<String> hasRevisions(Collection<Change> candidates)

Filter set of changes and see which of them really need to be updated. The changes that have their revision or better in the repo do not need update.

Parameters:

candidates - List of candidate changes

Returns:

Set of entity IDs for which the update is needed.

hasRevision

public boolean hasRevision(String entityId,
                           long revision)

Does the triple store have this revision or better.

fetchLeftOffTime

public Instant fetchLeftOffTime()

Fetch where we left off updating the repository.

Returns:

the date or null if we have nowhere to start from

updateLeftOffTime

public void updateLeftOffTime(Instant leftOffTime)

Update where we left off so when fetchLeftOffTime is next called it returns leftOffTime so we can continue from there after the updater is restarted.