Package org.dspace.statistics
Class SolrLoggerServiceImpl
- java.lang.Object
-
- org.dspace.statistics.SolrLoggerServiceImpl
-
- All Implemented Interfaces:
SolrLoggerService,org.springframework.beans.factory.InitializingBean
public class SolrLoggerServiceImpl extends Object implements SolrLoggerService, org.springframework.beans.factory.InitializingBean
Static holder for a HttpSolrClient connection pool to issue usage logging events to Solr from DSpace libraries, and some static query composers.- Author:
- ben at atmire.com, kevinvandevelde at atmire.com, mdiggory at atmire.com
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description classSolrLoggerServiceImpl.ResultProcessorstatic classSolrLoggerServiceImpl.StatisticsType
-
Field Summary
Fields Modifier and Type Field Description protected BitstreamServicebitstreamServiceprotected ContentServiceFactorycontentServiceFactorystatic StringDATE_FORMAT_8601static StringDATE_FORMAT_DCDATEprotected StringfilterQueryString of IP and Ranges in IPTable as a Solr Queryprotected com.maxmind.geoip2.DatabaseReaderlocationServiceprotected org.apache.solr.client.solrj.SolrClientsolrprotected booleanuseProxies
-
Constructor Summary
Constructors Modifier Constructor Description protectedSolrLoggerServiceImpl()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected voidaddAdditionalSolrYearCores(org.apache.solr.client.solrj.SolrQuery solrQuery)protected voidaddDocumentsToFile(Context context, org.apache.solr.common.SolrDocumentList docs, File exportOutput)voidafterPropertiesSet()protected org.apache.solr.client.solrj.impl.HttpSolrClientcreateCore(org.apache.solr.client.solrj.impl.HttpSolrClient solr, String coreName)voiddeleteIP(String ip)voiddeleteRobotsByIP()voiddeleteRobotsByIsBotFlag()voidexportHits()Export all SOLR usage statistics for viewing/downloading content to a flat text file.protected StringgenerateURL(String baseURL, Map<String,String> parameters)protected org.apache.solr.common.SolrInputDocumentgetCommonSolrDoc(DSpaceObject dspaceObject, String ip, String userAgent, String xforwardedfor, EPerson currentUser)protected org.apache.solr.common.SolrInputDocumentgetCommonSolrDoc(DSpaceObject dspaceObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser)Returns a solr input document containing common information about the statistics regardless if we are logging a search or a view of a DSpace objectprotected StringgetDateView(String name, String type, Context context)StringgetIgnoreSpiderIPs()Returns in a filterQuery string all the ip addresses that should be ignoredSet<String>getMultivaluedFieldNames()Retrieves a list of all the multi valued fields in the solr coreprotected voidinitSolrYearCores()booleanisUseProxies()voidmarkRobotByUserAgent(String agent)voidmarkRobotsByIP()voidoptimizeSOLR()Maintenance to keep a SOLR index efficient.voidpost(DSpaceObject dspaceObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser)Old post method, use the new postview method instead !voidpostSearch(DSpaceObject resultObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser, List<String> queries, int rpp, String sortBy, String order, int page, DSpaceObject scope)voidpostView(DSpaceObject dspaceObject, String ip, String userAgent, String xforwardedfor, EPerson currentUser)voidpostView(DSpaceObject dspaceObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser)Store a usage event into Solr.voidpostWorkflow(UsageWorkflowEvent usageWorkflowEvent)voidquery(String query, int max)org.apache.solr.client.solrj.response.QueryResponsequery(String query, String filterQuery, String facetField, int rows, int max, String dateType, String dateStart, String dateEnd, List<String> facetQueries, String sort, boolean ascending)ObjectCount[]queryFacetDate(String query, String filterQuery, int max, String dateType, String dateStart, String dateEnd, boolean showTotal, Context context)Query used to get values grouped by the date.ObjectCount[]queryFacetField(String query, String filterQuery, String facetField, int max, boolean showTotal, List<String> facetQueries)Query used to get values grouped by the given facet field.Map<String,Integer>queryFacetQuery(String query, String filterQuery, List<String> facetQueries)Map<String,List<String>>queryField(String query, List oldFieldVals, String field)ObjectCountqueryTotal(String query, String filterQuery)voidreindexBitstreamHits(boolean removeDeletedBitstreams)voidremoveIndex(String query)Delete data from the index, as described by a query.voidshardSolrIndex()voidstoreParents(org.apache.solr.common.SolrInputDocument doc1, DSpaceObject dso)Method just used to log the parents.voidupdate(String query, String action, List<String> fieldNames, List<List<Object>> fieldValuesList)
-
-
-
Field Detail
-
solr
protected org.apache.solr.client.solrj.SolrClient solr
-
DATE_FORMAT_8601
public static final String DATE_FORMAT_8601
- See Also:
- Constant Field Values
-
DATE_FORMAT_DCDATE
public static final String DATE_FORMAT_DCDATE
- See Also:
- Constant Field Values
-
locationService
protected com.maxmind.geoip2.DatabaseReader locationService
-
useProxies
protected boolean useProxies
-
bitstreamService
@Autowired(required=true) protected BitstreamService bitstreamService
-
contentServiceFactory
@Autowired(required=true) protected ContentServiceFactory contentServiceFactory
-
filterQuery
protected String filterQuery
String of IP and Ranges in IPTable as a Solr Query
-
-
Method Detail
-
afterPropertiesSet
public void afterPropertiesSet() throws Exception- Specified by:
afterPropertiesSetin interfaceorg.springframework.beans.factory.InitializingBean- Throws:
Exception
-
post
public void post(DSpaceObject dspaceObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser)
Description copied from interface:SolrLoggerServiceOld post method, use the new postview method instead !- Specified by:
postin interfaceSolrLoggerService- Parameters:
dspaceObject- the object used.request- the current request context.currentUser- the current session's user.
-
postView
public void postView(DSpaceObject dspaceObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser)
Description copied from interface:SolrLoggerServiceStore a usage event into Solr.- Specified by:
postViewin interfaceSolrLoggerService- Parameters:
dspaceObject- the object used.request- the current request context.currentUser- the current session's user.
-
postView
public void postView(DSpaceObject dspaceObject, String ip, String userAgent, String xforwardedfor, EPerson currentUser)
- Specified by:
postViewin interfaceSolrLoggerService
-
getCommonSolrDoc
protected org.apache.solr.common.SolrInputDocument getCommonSolrDoc(DSpaceObject dspaceObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser) throws SQLException
Returns a solr input document containing common information about the statistics regardless if we are logging a search or a view of a DSpace object- Parameters:
dspaceObject- the object used.request- the current request context.currentUser- the current session's user.- Returns:
- a solr input document
- Throws:
SQLException- in case of a database exception
-
getCommonSolrDoc
protected org.apache.solr.common.SolrInputDocument getCommonSolrDoc(DSpaceObject dspaceObject, String ip, String userAgent, String xforwardedfor, EPerson currentUser) throws SQLException
- Throws:
SQLException
-
postSearch
public void postSearch(DSpaceObject resultObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser, List<String> queries, int rpp, String sortBy, String order, int page, DSpaceObject scope)
- Specified by:
postSearchin interfaceSolrLoggerService
-
postWorkflow
public void postWorkflow(UsageWorkflowEvent usageWorkflowEvent) throws SQLException
- Specified by:
postWorkflowin interfaceSolrLoggerService- Throws:
SQLException
-
storeParents
public void storeParents(org.apache.solr.common.SolrInputDocument doc1, DSpaceObject dso) throws SQLExceptionDescription copied from interface:SolrLoggerServiceMethod just used to log the parents.- Community log: owning comms.
- Collection log: owning comms and their comms.
- Item log: owning colls/comms.
- Bitstream log: owning item/colls/comms.
- Specified by:
storeParentsin interfaceSolrLoggerService- Parameters:
doc1- the current SolrInputDocumentdso- the current dspace object we want to log- Throws:
SQLException- if database error ignore it
-
isUseProxies
public boolean isUseProxies()
- Specified by:
isUseProxiesin interfaceSolrLoggerService
-
removeIndex
public void removeIndex(String query) throws IOException, org.apache.solr.client.solrj.SolrServerException
Description copied from interface:SolrLoggerServiceDelete data from the index, as described by a query.- Specified by:
removeIndexin interfaceSolrLoggerService- Parameters:
query- description of the records to be deleted.- Throws:
IOException- A general class of exceptions produced by failed or interrupted I/O operations.org.apache.solr.client.solrj.SolrServerException- Exception from the Solr server to the solrj Java client.
-
queryField
public Map<String,List<String>> queryField(String query, List oldFieldVals, String field) throws IOException
- Specified by:
queryFieldin interfaceSolrLoggerService- Throws:
IOException
-
markRobotsByIP
public void markRobotsByIP()
- Specified by:
markRobotsByIPin interfaceSolrLoggerService
-
markRobotByUserAgent
public void markRobotByUserAgent(String agent)
- Specified by:
markRobotByUserAgentin interfaceSolrLoggerService
-
deleteRobotsByIsBotFlag
public void deleteRobotsByIsBotFlag()
- Specified by:
deleteRobotsByIsBotFlagin interfaceSolrLoggerService
-
deleteIP
public void deleteIP(String ip)
- Specified by:
deleteIPin interfaceSolrLoggerService
-
deleteRobotsByIP
public void deleteRobotsByIP()
- Specified by:
deleteRobotsByIPin interfaceSolrLoggerService
-
update
public void update(String query, String action, List<String> fieldNames, List<List<Object>> fieldValuesList) throws org.apache.solr.client.solrj.SolrServerException, IOException
- Specified by:
updatein interfaceSolrLoggerService- Throws:
org.apache.solr.client.solrj.SolrServerExceptionIOException
-
query
public void query(String query, int max) throws org.apache.solr.client.solrj.SolrServerException, IOException
- Specified by:
queryin interfaceSolrLoggerService- Throws:
org.apache.solr.client.solrj.SolrServerExceptionIOException
-
queryFacetField
public ObjectCount[] queryFacetField(String query, String filterQuery, String facetField, int max, boolean showTotal, List<String> facetQueries) throws org.apache.solr.client.solrj.SolrServerException, IOException
Description copied from interface:SolrLoggerServiceQuery used to get values grouped by the given facet field.- Specified by:
queryFacetFieldin interfaceSolrLoggerService- Parameters:
query- the query to be usedfilterQuery- filter queryfacetField- the facet field on which to group our valuesmax- the max number of values given back (in case of 10 the top 10 will be given)showTotal- a boolean determining whether the total amount should be given back as the last element of the arrayfacetQueries- list of facet queries- Returns:
- an array containing our results
- Throws:
org.apache.solr.client.solrj.SolrServerException- Exception from the Solr server to the solrj Java client.IOException- passed through.
-
queryFacetDate
public ObjectCount[] queryFacetDate(String query, String filterQuery, int max, String dateType, String dateStart, String dateEnd, boolean showTotal, Context context) throws org.apache.solr.client.solrj.SolrServerException, IOException
Description copied from interface:SolrLoggerServiceQuery used to get values grouped by the date.- Specified by:
queryFacetDatein interfaceSolrLoggerService- Parameters:
query- the query to be usedfilterQuery- filter querymax- the max number of values given back (in case of 10 the top 10 will be given)dateType- the type to be used (example: DAY, MONTH, YEAR)dateStart- the start date Format:(-3, -2, ..) the date is calculated relatively on todaydateEnd- the end date stop Format (-2, +1, ..) the date is calculated relatively on todayshowTotal- a boolean determining whether the total amount should be given back as the last element of the arraycontext- The relevant DSpace Context.- Returns:
- and array containing our results
- Throws:
org.apache.solr.client.solrj.SolrServerException- Exception from the Solr server to the solrj Java client.IOException- passed through.
-
queryFacetQuery
public Map<String,Integer> queryFacetQuery(String query, String filterQuery, List<String> facetQueries) throws org.apache.solr.client.solrj.SolrServerException, IOException
- Specified by:
queryFacetQueryin interfaceSolrLoggerService- Throws:
org.apache.solr.client.solrj.SolrServerExceptionIOException
-
queryTotal
public ObjectCount queryTotal(String query, String filterQuery) throws org.apache.solr.client.solrj.SolrServerException, IOException
- Specified by:
queryTotalin interfaceSolrLoggerService- Throws:
org.apache.solr.client.solrj.SolrServerExceptionIOException
-
query
public org.apache.solr.client.solrj.response.QueryResponse query(String query, String filterQuery, String facetField, int rows, int max, String dateType, String dateStart, String dateEnd, List<String> facetQueries, String sort, boolean ascending) throws org.apache.solr.client.solrj.SolrServerException, IOException
- Specified by:
queryin interfaceSolrLoggerService- Throws:
org.apache.solr.client.solrj.SolrServerExceptionIOException
-
getIgnoreSpiderIPs
public String getIgnoreSpiderIPs()
Description copied from interface:SolrLoggerServiceReturns in a filterQuery string all the ip addresses that should be ignored- Specified by:
getIgnoreSpiderIPsin interfaceSolrLoggerService- Returns:
- a string query with ip addresses
-
optimizeSOLR
public void optimizeSOLR()
Description copied from interface:SolrLoggerServiceMaintenance to keep a SOLR index efficient. Note: This might take a long time.- Specified by:
optimizeSOLRin interfaceSolrLoggerService
-
shardSolrIndex
public void shardSolrIndex() throws IOException, org.apache.solr.client.solrj.SolrServerException- Specified by:
shardSolrIndexin interfaceSolrLoggerService- Throws:
IOExceptionorg.apache.solr.client.solrj.SolrServerException
-
createCore
protected org.apache.solr.client.solrj.impl.HttpSolrClient createCore(org.apache.solr.client.solrj.impl.HttpSolrClient solr, String coreName) throws IOException, org.apache.solr.client.solrj.SolrServerException- Throws:
IOExceptionorg.apache.solr.client.solrj.SolrServerException
-
getMultivaluedFieldNames
public Set<String> getMultivaluedFieldNames() throws org.apache.solr.client.solrj.SolrServerException, IOException
Retrieves a list of all the multi valued fields in the solr core- Returns:
- all fields tagged as multivalued
- Throws:
org.apache.solr.client.solrj.SolrServerException- When getting the schema information from the SOLR core failsIOException- When connection to the SOLR server fails
-
reindexBitstreamHits
public void reindexBitstreamHits(boolean removeDeletedBitstreams) throws Exception- Specified by:
reindexBitstreamHitsin interfaceSolrLoggerService- Throws:
Exception
-
exportHits
public void exportHits() throws ExceptionDescription copied from interface:SolrLoggerServiceExport all SOLR usage statistics for viewing/downloading content to a flat text file. The file goes to a series- Specified by:
exportHitsin interfaceSolrLoggerService- Throws:
Exception- if error
-
addDocumentsToFile
protected void addDocumentsToFile(Context context, org.apache.solr.common.SolrDocumentList docs, File exportOutput) throws SQLException, ParseException, IOException
- Throws:
SQLExceptionParseExceptionIOException
-
generateURL
protected String generateURL(String baseURL, Map<String,String> parameters) throws UnsupportedEncodingException
- Throws:
UnsupportedEncodingException
-
addAdditionalSolrYearCores
protected void addAdditionalSolrYearCores(org.apache.solr.client.solrj.SolrQuery solrQuery)
-
initSolrYearCores
protected void initSolrYearCores()
-
-