Package org.dspace.statistics
Class SolrLoggerServiceImpl
- java.lang.Object
-
- org.dspace.statistics.SolrLoggerServiceImpl
-
- All Implemented Interfaces:
SolrLoggerService,org.springframework.beans.factory.InitializingBean
public class SolrLoggerServiceImpl extends Object implements SolrLoggerService, org.springframework.beans.factory.InitializingBean
Static holder for a HttpSolrClient connection pool to issue usage logging events to Solr from DSpace libraries, and some static query composers.- Author:
- ben at atmire.com, kevinvandevelde at atmire.com, mdiggory at atmire.com
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description classSolrLoggerServiceImpl.ResultProcessorstatic classSolrLoggerServiceImpl.StatisticsType
-
Field Summary
Fields Modifier and Type Field Description protected BitstreamServicebitstreamServiceprotected ContentServiceFactorycontentServiceFactorystatic StringDATE_FORMAT_8601static StringDATE_FORMAT_DCDATEprotected StringfilterQueryString of IP and Ranges in IPTable as a Solr Queryprotected com.maxmind.geoip2.DatabaseReaderlocationServiceprotected org.apache.solr.client.solrj.SolrClientsolrprotected booleanuseProxies
-
Constructor Summary
Constructors Modifier Constructor Description protectedSolrLoggerServiceImpl()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected voidaddAdditionalSolrYearCores(org.apache.solr.client.solrj.SolrQuery solrQuery)protected voidaddDocumentsToFile(Context context, org.apache.solr.common.SolrDocumentList docs, File exportOutput)voidafterPropertiesSet()ObjectanonymizeIp(String ip)Anonymize a given ipvoidcommit()Commit the solr core.protected org.apache.solr.client.solrj.impl.HttpSolrClientcreateCore(org.apache.solr.client.solrj.impl.HttpSolrClient solr, String coreName)voiddeleteIP(String ip)voiddeleteRobotsByIP()voiddeleteRobotsByIsBotFlag()voidexportHits()Export all SOLR usage statistics for viewing/downloading content to a flat text file.protected StringgenerateURL(String baseURL, Map<String,String> parameters)protected org.apache.solr.common.SolrInputDocumentgetCommonSolrDoc(DSpaceObject dspaceObject, String ip, String userAgent, String xforwardedfor, EPerson currentUser, String referrer)protected org.apache.solr.common.SolrInputDocumentgetCommonSolrDoc(DSpaceObject dspaceObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser)Returns a solr input document containing common information about the statistics regardless if we are logging a search or a view of a DSpace objectprotected org.apache.solr.common.SolrInputDocumentgetCommonSolrDoc(DSpaceObject dspaceObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser, String referrer)Returns a solr input document containing common information about the statistics regardless if we are logging a search or a view of a DSpace objectprotected StringgetDateView(String name, String type, Context context)StringgetIgnoreSpiderIPs()Returns in a filterQuery string all the ip addresses that should be ignoredSet<String>getMultivaluedFieldNames()Retrieves a list of all the multi valued fields in the solr core.protected voidinitSolrYearCores()booleanisUseProxies()voidmarkRobotByUserAgent(String agent)voidmarkRobotsByIP()voidpost(DSpaceObject dspaceObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser)Old post method, use the newSolrLoggerService.postView(org.dspace.content.DSpaceObject, javax.servlet.http.HttpServletRequest, org.dspace.eperson.EPerson)method instead !voidpostSearch(DSpaceObject resultObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser, List<String> queries, int rpp, String sortBy, String order, int page, DSpaceObject scope)voidpostView(DSpaceObject dspaceObject, String ip, String userAgent, String xforwardedfor, EPerson currentUser)voidpostView(DSpaceObject dspaceObject, String ip, String userAgent, String xforwardedfor, EPerson currentUser, String referrer)voidpostView(DSpaceObject dspaceObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser)Store a usage event into Solr.voidpostView(DSpaceObject dspaceObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser, String referrer)Store a usage event into Solr.voidpostWorkflow(UsageWorkflowEvent usageWorkflowEvent)voidquery(String query, int max, int facetMinCount)org.apache.solr.client.solrj.response.QueryResponsequery(String query, String filterQuery, String facetField, int rows, int max, String dateType, String dateStart, String dateEnd, List<String> facetQueries, String sort, boolean ascending, int facetMinCount)Perform a solr query.org.apache.solr.client.solrj.response.QueryResponsequery(String query, String filterQuery, String facetField, int rows, int max, String dateType, String dateStart, String dateEnd, List<String> facetQueries, String sort, boolean ascending, int facetMinCount, boolean defaultFilterQueries)Perform a solr query.ObjectCount[]queryFacetDate(String query, String filterQuery, int max, String dateType, String dateStart, String dateEnd, boolean showTotal, Context context, int facetMinCount)Query used to get values grouped by the date.ObjectCount[]queryFacetField(String query, String filterQuery, String facetField, int max, boolean showTotal, List<String> facetQueries, int facetMinCount)Query used to get values grouped by the given facet field.Map<String,Integer>queryFacetQuery(String query, String filterQuery, List<String> facetQueries, int facetMinCount)Map<String,List<String>>queryField(String query, List oldFieldVals, String field)ObjectCountqueryTotal(String query, String filterQuery, int facetMinCount)voidreindexBitstreamHits(boolean removeDeletedBitstreams)voidremoveIndex(String query)Delete data from the index, as described by a query.voidshardSolrIndex()voidstoreParents(org.apache.solr.common.SolrInputDocument doc1, DSpaceObject dso)Method just used to log the parents.voidupdate(String query, String action, List<String> fieldNames, List<List<Object>> fieldValuesList)voidupdate(String query, String action, List<String> fieldNames, List<List<Object>> fieldValuesList, boolean commit)Update the solr core.
-
-
-
Field Detail
-
solr
protected org.apache.solr.client.solrj.SolrClient solr
-
DATE_FORMAT_8601
public static final String DATE_FORMAT_8601
- See Also:
- Constant Field Values
-
DATE_FORMAT_DCDATE
public static final String DATE_FORMAT_DCDATE
- See Also:
- Constant Field Values
-
locationService
protected com.maxmind.geoip2.DatabaseReader locationService
-
useProxies
protected boolean useProxies
-
bitstreamService
@Autowired(required=true) protected BitstreamService bitstreamService
-
contentServiceFactory
@Autowired(required=true) protected ContentServiceFactory contentServiceFactory
-
filterQuery
protected String filterQuery
String of IP and Ranges in IPTable as a Solr Query
-
-
Method Detail
-
afterPropertiesSet
public void afterPropertiesSet() throws Exception- Specified by:
afterPropertiesSetin interfaceorg.springframework.beans.factory.InitializingBean- Throws:
Exception
-
post
public void post(DSpaceObject dspaceObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser)
Description copied from interface:SolrLoggerServiceOld post method, use the newSolrLoggerService.postView(org.dspace.content.DSpaceObject, javax.servlet.http.HttpServletRequest, org.dspace.eperson.EPerson)method instead !- Specified by:
postin interfaceSolrLoggerService- Parameters:
dspaceObject- the object used.request- the current request context.currentUser- the current session's user.
-
postView
public void postView(DSpaceObject dspaceObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser)
Description copied from interface:SolrLoggerServiceStore a usage event into Solr.- Specified by:
postViewin interfaceSolrLoggerService- Parameters:
dspaceObject- the object used.request- the current request context.currentUser- the current session's user.
-
postView
public void postView(DSpaceObject dspaceObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser, String referrer)
Description copied from interface:SolrLoggerServiceStore a usage event into Solr.- Specified by:
postViewin interfaceSolrLoggerService- Parameters:
dspaceObject- the object used.request- the current request context.currentUser- the current session's user.referrer- the optional referrer.
-
postView
public void postView(DSpaceObject dspaceObject, String ip, String userAgent, String xforwardedfor, EPerson currentUser)
- Specified by:
postViewin interfaceSolrLoggerService
-
postView
public void postView(DSpaceObject dspaceObject, String ip, String userAgent, String xforwardedfor, EPerson currentUser, String referrer)
- Specified by:
postViewin interfaceSolrLoggerService
-
getCommonSolrDoc
protected org.apache.solr.common.SolrInputDocument getCommonSolrDoc(DSpaceObject dspaceObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser) throws SQLException
Returns a solr input document containing common information about the statistics regardless if we are logging a search or a view of a DSpace object- Parameters:
dspaceObject- the object used.request- the current request context.currentUser- the current session's user.- Returns:
- a solr input document
- Throws:
SQLException- in case of a database exception
-
getCommonSolrDoc
protected org.apache.solr.common.SolrInputDocument getCommonSolrDoc(DSpaceObject dspaceObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser, String referrer) throws SQLException
Returns a solr input document containing common information about the statistics regardless if we are logging a search or a view of a DSpace object- Parameters:
dspaceObject- the object used.request- the current request context.currentUser- the current session's user.referrer- the optional referrer.- Returns:
- a solr input document
- Throws:
SQLException- in case of a database exception
-
getCommonSolrDoc
protected org.apache.solr.common.SolrInputDocument getCommonSolrDoc(DSpaceObject dspaceObject, String ip, String userAgent, String xforwardedfor, EPerson currentUser, String referrer) throws SQLException
- Throws:
SQLException
-
postSearch
public void postSearch(DSpaceObject resultObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser, List<String> queries, int rpp, String sortBy, String order, int page, DSpaceObject scope)
- Specified by:
postSearchin interfaceSolrLoggerService
-
postWorkflow
public void postWorkflow(UsageWorkflowEvent usageWorkflowEvent) throws SQLException
- Specified by:
postWorkflowin interfaceSolrLoggerService- Throws:
SQLException
-
storeParents
public void storeParents(org.apache.solr.common.SolrInputDocument doc1, DSpaceObject dso) throws SQLExceptionDescription copied from interface:SolrLoggerServiceMethod just used to log the parents.- Community log: owning comms.
- Collection log: owning comms and their comms.
- Item log: owning colls/comms.
- Bitstream log: owning item/colls/comms.
- Specified by:
storeParentsin interfaceSolrLoggerService- Parameters:
doc1- the current SolrInputDocumentdso- the current dspace object we want to log- Throws:
SQLException- if database error ignore it
-
isUseProxies
public boolean isUseProxies()
- Specified by:
isUseProxiesin interfaceSolrLoggerService
-
removeIndex
public void removeIndex(String query) throws IOException, org.apache.solr.client.solrj.SolrServerException
Description copied from interface:SolrLoggerServiceDelete data from the index, as described by a query.- Specified by:
removeIndexin interfaceSolrLoggerService- Parameters:
query- description of the records to be deleted.- Throws:
IOException- A general class of exceptions produced by failed or interrupted I/O operations.org.apache.solr.client.solrj.SolrServerException- Exception from the Solr server to the solrj Java client.
-
queryField
public Map<String,List<String>> queryField(String query, List oldFieldVals, String field) throws IOException
- Specified by:
queryFieldin interfaceSolrLoggerService- Throws:
IOException
-
markRobotsByIP
public void markRobotsByIP()
- Specified by:
markRobotsByIPin interfaceSolrLoggerService
-
markRobotByUserAgent
public void markRobotByUserAgent(String agent)
- Specified by:
markRobotByUserAgentin interfaceSolrLoggerService
-
deleteRobotsByIsBotFlag
public void deleteRobotsByIsBotFlag()
- Specified by:
deleteRobotsByIsBotFlagin interfaceSolrLoggerService
-
deleteIP
public void deleteIP(String ip)
- Specified by:
deleteIPin interfaceSolrLoggerService
-
deleteRobotsByIP
public void deleteRobotsByIP()
- Specified by:
deleteRobotsByIPin interfaceSolrLoggerService
-
update
public void update(String query, String action, List<String> fieldNames, List<List<Object>> fieldValuesList) throws org.apache.solr.client.solrj.SolrServerException, IOException
- Specified by:
updatein interfaceSolrLoggerService- Throws:
org.apache.solr.client.solrj.SolrServerExceptionIOException
-
update
public void update(String query, String action, List<String> fieldNames, List<List<Object>> fieldValuesList, boolean commit) throws org.apache.solr.client.solrj.SolrServerException, IOException
Description copied from interface:SolrLoggerServiceUpdate the solr core.- Specified by:
updatein interfaceSolrLoggerService- Parameters:
query- query indicating which documents to updateaction- the update action keywordfieldNames- the fields to updatefieldValuesList- the values for the fields to updatecommit- whether to commit the changes- Throws:
org.apache.solr.client.solrj.SolrServerExceptionIOException
-
query
public void query(String query, int max, int facetMinCount) throws org.apache.solr.client.solrj.SolrServerException, IOException
- Specified by:
queryin interfaceSolrLoggerService- Throws:
org.apache.solr.client.solrj.SolrServerExceptionIOException
-
queryFacetField
public ObjectCount[] queryFacetField(String query, String filterQuery, String facetField, int max, boolean showTotal, List<String> facetQueries, int facetMinCount) throws org.apache.solr.client.solrj.SolrServerException, IOException
Description copied from interface:SolrLoggerServiceQuery used to get values grouped by the given facet field.- Specified by:
queryFacetFieldin interfaceSolrLoggerService- Parameters:
query- the query to be usedfilterQuery- filter queryfacetField- the facet field on which to group our valuesmax- the max number of values given back (in case of 10 the top 10 will be given)showTotal- a boolean determining whether the total amount should be given back as the last element of the arrayfacetQueries- list of facet queriesfacetMinCount- Minimum count of results facet must have to return a result- Returns:
- an array containing our results
- Throws:
org.apache.solr.client.solrj.SolrServerException- Exception from the Solr server to the solrj Java client.IOException- passed through.
-
queryFacetDate
public ObjectCount[] queryFacetDate(String query, String filterQuery, int max, String dateType, String dateStart, String dateEnd, boolean showTotal, Context context, int facetMinCount) throws org.apache.solr.client.solrj.SolrServerException, IOException
Description copied from interface:SolrLoggerServiceQuery used to get values grouped by the date.- Specified by:
queryFacetDatein interfaceSolrLoggerService- Parameters:
query- the query to be usedfilterQuery- filter querymax- the max number of values given back (in case of 10 the top 10 will be given)dateType- the type to be used (example: DAY, MONTH, YEAR)dateStart- the start date Format:(-3, -2, ..) the date is calculated relatively on todaydateEnd- the end date stop Format (-2, +1, ..) the date is calculated relatively on todayshowTotal- a boolean determining whether the total amount should be given back as the last element of the arraycontext- The relevant DSpace Context.facetMinCount- Minimum count of results facet must have to return a result- Returns:
- and array containing our results
- Throws:
org.apache.solr.client.solrj.SolrServerException- Exception from the Solr server to the solrj Java client.IOException- passed through.
-
queryFacetQuery
public Map<String,Integer> queryFacetQuery(String query, String filterQuery, List<String> facetQueries, int facetMinCount) throws org.apache.solr.client.solrj.SolrServerException, IOException
- Specified by:
queryFacetQueryin interfaceSolrLoggerService- Throws:
org.apache.solr.client.solrj.SolrServerExceptionIOException
-
queryTotal
public ObjectCount queryTotal(String query, String filterQuery, int facetMinCount) throws org.apache.solr.client.solrj.SolrServerException, IOException
- Specified by:
queryTotalin interfaceSolrLoggerService- Throws:
org.apache.solr.client.solrj.SolrServerExceptionIOException
-
query
public org.apache.solr.client.solrj.response.QueryResponse query(String query, String filterQuery, String facetField, int rows, int max, String dateType, String dateStart, String dateEnd, List<String> facetQueries, String sort, boolean ascending, int facetMinCount) throws org.apache.solr.client.solrj.SolrServerException, IOException
Description copied from interface:SolrLoggerServicePerform a solr query.- Specified by:
queryin interfaceSolrLoggerService- Parameters:
query- the query to be usedfilterQuery- filter queryfacetField- field to facet the results byrows- the max number of results to returnmax- the max number of facets to returndateType- the type to be used (example: DAY, MONTH, YEAR)dateStart- the start date Format:(-3, -2, ..) the date is calculated relatively on todaydateEnd- the end date stop Format (-2, +1, ..) the date is calculated relatively on todayfacetQueries- list of facet queriessort- the sort fieldascending- the sort direction (true: ascending)facetMinCount- Minimum count of results facet must have to return a result- Throws:
org.apache.solr.client.solrj.SolrServerException- Exception from the Solr server to the solrj Java client.IOException- passed through.
-
query
public org.apache.solr.client.solrj.response.QueryResponse query(String query, String filterQuery, String facetField, int rows, int max, String dateType, String dateStart, String dateEnd, List<String> facetQueries, String sort, boolean ascending, int facetMinCount, boolean defaultFilterQueries) throws org.apache.solr.client.solrj.SolrServerException, IOException
Description copied from interface:SolrLoggerServicePerform a solr query.- Specified by:
queryin interfaceSolrLoggerService- Parameters:
query- the query to be usedfilterQuery- filter queryfacetField- field to facet the results byrows- the max number of results to returnmax- the max number of facets to returndateType- the type to be used (example: DAY, MONTH, YEAR)dateStart- the start date Format:(-3, -2, ..) the date is calculated relatively on todaydateEnd- the end date stop Format (-2, +1, ..) the date is calculated relatively on todayfacetQueries- list of facet queriessort- the sort fieldascending- the sort direction (true: ascending)facetMinCount- Minimum count of results facet must have to return a resultdefaultFilterQueries- use the default filter queries- Throws:
org.apache.solr.client.solrj.SolrServerException- Exception from the Solr server to the solrj Java client.IOException- passed through.
-
getIgnoreSpiderIPs
public String getIgnoreSpiderIPs()
Description copied from interface:SolrLoggerServiceReturns in a filterQuery string all the ip addresses that should be ignored- Specified by:
getIgnoreSpiderIPsin interfaceSolrLoggerService- Returns:
- a string query with ip addresses
-
shardSolrIndex
public void shardSolrIndex() throws IOException, org.apache.solr.client.solrj.SolrServerException- Specified by:
shardSolrIndexin interfaceSolrLoggerService- Throws:
IOExceptionorg.apache.solr.client.solrj.SolrServerException
-
createCore
protected org.apache.solr.client.solrj.impl.HttpSolrClient createCore(org.apache.solr.client.solrj.impl.HttpSolrClient solr, String coreName) throws IOException, org.apache.solr.client.solrj.SolrServerException- Throws:
IOExceptionorg.apache.solr.client.solrj.SolrServerException
-
getMultivaluedFieldNames
public Set<String> getMultivaluedFieldNames() throws org.apache.solr.client.solrj.SolrServerException, IOException
Retrieves a list of all the multi valued fields in the solr core.- Returns:
- all fields tagged as multivalued
- Throws:
org.apache.solr.client.solrj.SolrServerException- When getting the schema information from the SOLR core failsIOException- When connection to the SOLR server fails
-
reindexBitstreamHits
public void reindexBitstreamHits(boolean removeDeletedBitstreams) throws Exception- Specified by:
reindexBitstreamHitsin interfaceSolrLoggerService- Throws:
Exception
-
exportHits
public void exportHits() throws ExceptionDescription copied from interface:SolrLoggerServiceExport all SOLR usage statistics for viewing/downloading content to a flat text file. The file goes to a series- Specified by:
exportHitsin interfaceSolrLoggerService- Throws:
Exception- if error
-
commit
public void commit() throws IOException, org.apache.solr.client.solrj.SolrServerExceptionDescription copied from interface:SolrLoggerServiceCommit the solr core.- Specified by:
commitin interfaceSolrLoggerService- Throws:
IOExceptionorg.apache.solr.client.solrj.SolrServerException
-
addDocumentsToFile
protected void addDocumentsToFile(Context context, org.apache.solr.common.SolrDocumentList docs, File exportOutput) throws SQLException, ParseException, IOException
- Throws:
SQLExceptionParseExceptionIOException
-
generateURL
protected String generateURL(String baseURL, Map<String,String> parameters) throws UnsupportedEncodingException
- Throws:
UnsupportedEncodingException
-
addAdditionalSolrYearCores
protected void addAdditionalSolrYearCores(org.apache.solr.client.solrj.SolrQuery solrQuery)
-
initSolrYearCores
protected void initSolrYearCores()
-
anonymizeIp
public Object anonymizeIp(String ip) throws UnknownHostException
Description copied from interface:SolrLoggerServiceAnonymize a given ip- Specified by:
anonymizeIpin interfaceSolrLoggerService- Parameters:
ip- The ip to anonymize.- Throws:
UnknownHostException
-
-