org.wikidata.query.rdf.tool.rdf

Class RdfRepository

java.lang.Object

org.wikidata.query.rdf.tool.rdf.RdfRepository

public class RdfRepository
extends Object

Wrapper for communicating with the RDF repository.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	RdfRepository.UpdateCountResponse Attempts to log update response information but very likely only works for Blazegraph.

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.wikidata.query.rdf.tool.rdf.RdfRepository.ResponseHandler<Boolean>	ASK_QUERY_RESPONSE Parse the response from an ask query into a boolean.
protected static org.wikidata.query.rdf.tool.rdf.RdfRepository.ResponseHandler<org.openrdf.query.TupleQueryResult>	TUPLE_QUERY_RESPONSE Parse the response from a regular query into a TupleQueryResult.
protected static org.wikidata.query.rdf.tool.rdf.RdfRepository.ResponseHandler<Integer>	UPDATE_COUNT_RESPONSE Count and log the number of updates.

Constructor Summary

Constructors

Constructor and Description
RdfRepository(URI uri, WikibaseUris uris)

Method Summary

Modifier and Type	Method and Description
boolean	ask(String sparql) Execute a SPARQL ask and parse the boolean result.
protected org.apache.http.impl.client.CloseableHttpClient	client() Allow subclass access to the HTTP client.
protected <T> T	execute(String type, org.wikidata.query.rdf.tool.rdf.RdfRepository.ResponseHandler<T> responseHandler, String sparql) Execute some raw SPARQL.
Date	fetchLeftOffTime() Fetch where we left off updating the repository.
int	getDelay() Get retry delay.
int	getMaxRetries() Get max retries count.
com.google.common.collect.Multimap<String,String>	getRefs(Collection<String> entityIds) Get list of reference subjects connected to entity.
String	getSyncQuery(String entityId, Collection<org.openrdf.model.Statement> statements, Collection<String> valueList) Provides the SPARQL needed to synchronize the data statements.
com.google.common.collect.Multimap<String,String>	getValues(Collection<String> entityIds) Get list of value subjects connected to entity.
boolean	hasRevision(String entityId, long revision) Does the triple store have this revision or better.
Set<String>	hasRevisions(Collection<Change> candidates) Filter set of changes and see which of them really need to be updated.
org.openrdf.query.TupleQueryResult	query(String sparql) Execute some SPARQL which returns a results table.
protected String	responseBodyAsString(org.apache.http.client.methods.CloseableHttpResponse response) Fetch the body of the response as a string.
RdfRepository	setDelay(int delay) Set retry delay.
RdfRepository	setMaxRetries(int maxRetries) Set how many times we retry a failed HTTP call.
int	sync(String entityId, Collection<org.openrdf.model.Statement> statements) Synchronizes the RDF repository's representation.
int	sync(String entityId, Collection<org.openrdf.model.Statement> statements, Collection<String> valueList) Synchronizes the RDF repository's representation of an entity to be exactly the provided statements.
int	syncFromChanges(Collection<Change> changes, boolean verifyResult) Sync repository from changes list.
int	syncQuery(String query) Synchronizes the RDF repository's representation of an entity to be exactly the provided statements.
void	updateLeftOffTime(Date leftOffTime) Update where we left off so when fetchLeftOffTime is next called it returns leftOffTime so we can continue from there after the updater is restarted.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

UPDATE_COUNT_RESPONSE

protected static final org.wikidata.query.rdf.tool.rdf.RdfRepository.ResponseHandler<Integer> UPDATE_COUNT_RESPONSE

Count and log the number of updates.

TUPLE_QUERY_RESPONSE

protected static final org.wikidata.query.rdf.tool.rdf.RdfRepository.ResponseHandler<org.openrdf.query.TupleQueryResult> TUPLE_QUERY_RESPONSE

Parse the response from a regular query into a TupleQueryResult.

ASK_QUERY_RESPONSE

protected static final org.wikidata.query.rdf.tool.rdf.RdfRepository.ResponseHandler<Boolean> ASK_QUERY_RESPONSE

Parse the response from an ask query into a boolean.

Constructor Detail

RdfRepository

public RdfRepository(URI uri,
                     WikibaseUris uris)

Method Detail

client

protected org.apache.http.impl.client.CloseableHttpClient client()

Allow subclass access to the HTTP client.

getMaxRetries

public int getMaxRetries()

Get max retries count.

Returns:

How many times we retry a failed HTTP call.

setMaxRetries

public RdfRepository setMaxRetries(int maxRetries)

Set how many times we retry a failed HTTP call.

Returns:

this

getDelay

public int getDelay()

Get retry delay.

Returns:

How long to delay after failing first HTTP call, in milliseconds.

setDelay

public RdfRepository setDelay(int delay)

Set retry delay. Specifies how long to delay after failing first HTTP call, in milliseconds. Next retries would be slower by 2x, 3x, 4x etc. until maxRetries is exhausted.

Returns:

The repository object

getValues

public com.google.common.collect.Multimap<String,String> getValues(Collection<String> entityIds)

Get list of value subjects connected to entity. The connection is either via statement or via reference or via qualifier.

Parameters:

entityIds -

Returns:

Set of value subjects

getRefs

public com.google.common.collect.Multimap<String,String> getRefs(Collection<String> entityIds)

Get list of reference subjects connected to entity.

Parameters:

entityIds -

Returns:

Set of references

getSyncQuery

public String getSyncQuery(String entityId,
                           Collection<org.openrdf.model.Statement> statements,
                           Collection<String> valueList)

Provides the SPARQL needed to synchronize the data statements.

Parameters:

entityId - id of the entity to sync

statements - all known statements about the entity

valueList - list of used values, for cleanup

Returns:

the number of statements modified

syncFromChanges

public int syncFromChanges(Collection<Change> changes,
                           boolean verifyResult)

Sync repository from changes list.

Parameters:

changes - List of changes.

Returns:

Number of triples modified.

sync

public int sync(String entityId,
                Collection<org.openrdf.model.Statement> statements,
                Collection<String> valueList)

Synchronizes the RDF repository's representation of an entity to be exactly the provided statements. You can think of the RDF managed for an entity as a tree rooted at the entity. The managed tree ends where the next entity's managed tree starts. For example Q23 from wikidata includes all statements about George Washington but not those about Martha (Q191789) even though she is linked by the spouse attribute. On the other hand the qualifiers on statements about George are included in George.

Parameters:

entityId - id of the entity to sync

statements - all known statements about the entity

valueList - list of used values, for cleanup

Returns:

the number of statements modified

syncQuery

public int syncQuery(String query)

Synchronizes the RDF repository's representation of an entity to be exactly the provided statements.

Parameters:

query - Query text

Returns:

the number of statements modified

sync

public int sync(String entityId,
                Collection<org.openrdf.model.Statement> statements)

Synchronizes the RDF repository's representation. See also: sync(String, Collection, Collection)

Parameters:

entityId - id of the entity to sync

statements - all known statements about the entity

Returns:

the number of statements modified

hasRevisions

public Set<String> hasRevisions(Collection<Change> candidates)

Filter set of changes and see which of them really need to be updated. The changes that have their revision or better in the repo do not need update.

Parameters:

candidates - List of candidate changes

Returns:

Set of entity IDs for which the update is needed.

hasRevision

public boolean hasRevision(String entityId,
                           long revision)

Does the triple store have this revision or better.

fetchLeftOffTime

public Date fetchLeftOffTime()

Fetch where we left off updating the repository.

Returns:

the date or null if we have nowhere to start from

updateLeftOffTime

public void updateLeftOffTime(Date leftOffTime)

Update where we left off so when fetchLeftOffTime is next called it returns leftOffTime so we can continue from there after the updater is restarted.

ask

public boolean ask(String sparql)

Execute a SPARQL ask and parse the boolean result.

query

public org.openrdf.query.TupleQueryResult query(String sparql)

Execute some SPARQL which returns a results table.

execute

protected <T> T execute(String type,
                        org.wikidata.query.rdf.tool.rdf.RdfRepository.ResponseHandler<T> responseHandler,
                        String sparql)

Execute some raw SPARQL.

Parameters:

type - name of the parameter in which to send sparql

Returns:

results string from the server

responseBodyAsString

protected String responseBodyAsString(org.apache.http.client.methods.CloseableHttpResponse response)
                               throws IOException

Fetch the body of the response as a string.

Throws:

IOException - if there is an error reading the response