org.dspace.statistics

Class SolrLoggerServiceImpl

java.lang.Object

org.dspace.statistics.SolrLoggerServiceImpl

All Implemented Interfaces:

SolrLoggerService, org.springframework.beans.factory.InitializingBean

public class SolrLoggerServiceImpl
extends Object
implements SolrLoggerService, org.springframework.beans.factory.InitializingBean

Static holder for a HttpSolrClient connection pool to issue usage logging events to Solr from DSpace libraries, and some static query composers.

Author:

ben at atmire.com, kevinvandevelde at atmire.com, mdiggory at atmire.com

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	SolrLoggerServiceImpl.ResultProcessor
static class	SolrLoggerServiceImpl.StatisticsType

Field Summary

Fields

Modifier and Type	Field and Description
protected BitstreamService	bitstreamService
protected ContentServiceFactory	contentServiceFactory
static String	DATE_FORMAT_8601
static String	DATE_FORMAT_DCDATE
protected String	filterQuery String of IP and Ranges in IPTable as a Solr Query
protected com.maxmind.geoip.LookupService	locationService
protected org.apache.solr.client.solrj.impl.HttpSolrServer	solr
protected boolean	useProxies

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	SolrLoggerServiceImpl()

Method Summary

Methods

Modifier and Type	Method and Description
protected void	addAdditionalSolrYearCores(org.apache.solr.client.solrj.SolrQuery solrQuery)
protected void	addDocumentsToFile(Context context, org.apache.solr.common.SolrDocumentList docs, File exportOutput)
void	afterPropertiesSet()
protected org.apache.solr.client.solrj.impl.HttpSolrServer	createCore(org.apache.solr.client.solrj.impl.HttpSolrServer solr, String coreName)
void	deleteIP(String ip)
void	deleteRobotsByIP()
void	deleteRobotsByIsBotFlag()
void	exportHits() Export all SOLR usage statistics for viewing/downloading content to a flat text file.
protected String	generateURL(String baseURL, Map<String,String> parameters)
protected org.apache.solr.common.SolrInputDocument	getCommonSolrDoc(DSpaceObject dspaceObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser) Returns a solr input document containing common information about the statistics regardless if we are logging a search or a view of a DSpace object
protected org.apache.solr.common.SolrInputDocument	getCommonSolrDoc(DSpaceObject dspaceObject, String ip, String userAgent, String xforwardedfor, EPerson currentUser)
protected String	getDateView(String name, String type, Context context)
String	getIgnoreSpiderIPs() Returns in a filterQuery string all the ip addresses that should be ignored
boolean	isUseProxies()
void	markRobotByUserAgent(String agent)
void	markRobotsByIP()
void	optimizeSOLR() Maintenance to keep a SOLR index efficient.
void	post(DSpaceObject dspaceObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser) Old post method, use the new postview method instead !
void	postSearch(DSpaceObject resultObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser, List<String> queries, int rpp, String sortBy, String order, int page, DSpaceObject scope)
void	postView(DSpaceObject dspaceObject, javax.servlet.http.HttpServletRequest request, EPerson currentUser) Store a usage event into Solr.
void	postView(DSpaceObject dspaceObject, String ip, String userAgent, String xforwardedfor, EPerson currentUser)
void	postWorkflow(UsageWorkflowEvent usageWorkflowEvent)
void	query(String query, int max)
org.apache.solr.client.solrj.response.QueryResponse	query(String query, String filterQuery, String facetField, int rows, int max, String dateType, String dateStart, String dateEnd, List<String> facetQueries, String sort, boolean ascending)
ObjectCount[]	queryFacetDate(String query, String filterQuery, int max, String dateType, String dateStart, String dateEnd, boolean showTotal, Context context) Query used to get values grouped by the date.
ObjectCount[]	queryFacetField(String query, String filterQuery, String facetField, int max, boolean showTotal, List<String> facetQueries) Query used to get values grouped by the given facet field.
Map<String,Integer>	queryFacetQuery(String query, String filterQuery, List<String> facetQueries)
Map<String,List<String>>	queryField(String query, List oldFieldVals, String field)
ObjectCount	queryTotal(String query, String filterQuery)
void	reindexBitstreamHits(boolean removeDeletedBitstreams)
void	removeIndex(String query) Delete data from the index, as described by a query.
void	shardSolrIndex()
void	storeParents(org.apache.solr.common.SolrInputDocument doc1, DSpaceObject dso) Method just used to log the parents.
void	update(String query, String action, List<String> fieldNames, List<List<Object>> fieldValuesList)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

solr

protected org.apache.solr.client.solrj.impl.HttpSolrServer solr

DATE_FORMAT_8601

public static final String DATE_FORMAT_8601

See Also:

Constant Field Values

DATE_FORMAT_DCDATE

public static final String DATE_FORMAT_DCDATE

See Also:

Constant Field Values

locationService

protected com.maxmind.geoip.LookupService locationService

useProxies

protected boolean useProxies

bitstreamService

@Autowired(required=true)
protected BitstreamService bitstreamService

contentServiceFactory

@Autowired(required=true)
protected ContentServiceFactory contentServiceFactory

filterQuery

protected String filterQuery

String of IP and Ranges in IPTable as a Solr Query

Constructor Detail

SolrLoggerServiceImpl

protected SolrLoggerServiceImpl()

Method Detail

afterPropertiesSet

public void afterPropertiesSet()
                        throws Exception

Specified by:

afterPropertiesSet in interface org.springframework.beans.factory.InitializingBean

Throws:

Exception

post

public void post(DSpaceObject dspaceObject,
        javax.servlet.http.HttpServletRequest request,
        EPerson currentUser)

Description copied from interface: SolrLoggerService

Old post method, use the new postview method instead !

Specified by:

post in interface SolrLoggerService

Parameters:

dspaceObject - the object used.

request - the current request context.

currentUser - the current session's user.

postView

public void postView(DSpaceObject dspaceObject,
            javax.servlet.http.HttpServletRequest request,
            EPerson currentUser)

Description copied from interface: SolrLoggerService

Store a usage event into Solr.

Specified by:

postView in interface SolrLoggerService

Parameters:

dspaceObject - the object used.

request - the current request context.

currentUser - the current session's user.

postView

public void postView(DSpaceObject dspaceObject,
            String ip,
            String userAgent,
            String xforwardedfor,
            EPerson currentUser)

Specified by:

postView in interface SolrLoggerService

getCommonSolrDoc

protected org.apache.solr.common.SolrInputDocument getCommonSolrDoc(DSpaceObject dspaceObject,
                                                        javax.servlet.http.HttpServletRequest request,
                                                        EPerson currentUser)
                                                             throws SQLException

Returns a solr input document containing common information about the statistics regardless if we are logging a search or a view of a DSpace object

Parameters:

dspaceObject - the object used.

request - the current request context.

currentUser - the current session's user.

Returns:

a solr input document

Throws:

SQLException - in case of a database exception

getCommonSolrDoc

protected org.apache.solr.common.SolrInputDocument getCommonSolrDoc(DSpaceObject dspaceObject,
                                                        String ip,
                                                        String userAgent,
                                                        String xforwardedfor,
                                                        EPerson currentUser)
                                                             throws SQLException

Throws:

SQLException

postSearch

public void postSearch(DSpaceObject resultObject,
              javax.servlet.http.HttpServletRequest request,
              EPerson currentUser,
              List<String> queries,
              int rpp,
              String sortBy,
              String order,
              int page,
              DSpaceObject scope)

Specified by:

postSearch in interface SolrLoggerService

postWorkflow

public void postWorkflow(UsageWorkflowEvent usageWorkflowEvent)
                  throws SQLException

Specified by:

postWorkflow in interface SolrLoggerService

Throws:

SQLException

storeParents

public void storeParents(org.apache.solr.common.SolrInputDocument doc1,
                DSpaceObject dso)
                  throws SQLException

Description copied from interface: SolrLoggerService

Method just used to log the parents.

• Community log: owning comms.
• Collection log: owning comms & their comms.
• Item log: owning colls/comms.
• Bitstream log: owning item/colls/comms.

Specified by:

storeParents in interface SolrLoggerService

Parameters:

doc1 - the current SolrInputDocument

dso - the current dspace object we want to log

Throws:

SQLException - ignore it

isUseProxies

public boolean isUseProxies()

Specified by:

isUseProxies in interface SolrLoggerService

removeIndex

public void removeIndex(String query)
                 throws IOException,
                        org.apache.solr.client.solrj.SolrServerException

Description copied from interface: SolrLoggerService

Delete data from the index, as described by a query.

Specified by:

removeIndex in interface SolrLoggerService

Parameters:

query - description of the records to be deleted.

Throws:

IOException

org.apache.solr.client.solrj.SolrServerException

queryField

public Map<String,List<String>> queryField(String query,
                                  List oldFieldVals,
                                  String field)

Specified by:

queryField in interface SolrLoggerService

markRobotsByIP

public void markRobotsByIP()

Specified by:

markRobotsByIP in interface SolrLoggerService

markRobotByUserAgent

public void markRobotByUserAgent(String agent)

Specified by:

markRobotByUserAgent in interface SolrLoggerService

deleteRobotsByIsBotFlag

public void deleteRobotsByIsBotFlag()

Specified by:

deleteRobotsByIsBotFlag in interface SolrLoggerService

deleteIP

public void deleteIP(String ip)

Specified by:

deleteIP in interface SolrLoggerService

deleteRobotsByIP

public void deleteRobotsByIP()

Specified by:

deleteRobotsByIP in interface SolrLoggerService

update

public void update(String query,
          String action,
          List<String> fieldNames,
          List<List<Object>> fieldValuesList)
            throws org.apache.solr.client.solrj.SolrServerException,
                   IOException

Specified by:

update in interface SolrLoggerService

Throws:

org.apache.solr.client.solrj.SolrServerException

IOException

query

public void query(String query,
         int max)
           throws org.apache.solr.client.solrj.SolrServerException

Specified by:

query in interface SolrLoggerService

Throws:

org.apache.solr.client.solrj.SolrServerException

queryFacetField

public ObjectCount[] queryFacetField(String query,
                            String filterQuery,
                            String facetField,
                            int max,
                            boolean showTotal,
                            List<String> facetQueries)
                              throws org.apache.solr.client.solrj.SolrServerException

Description copied from interface: SolrLoggerService

Query used to get values grouped by the given facet field.

Specified by:

queryFacetField in interface SolrLoggerService

Parameters:

query - the query to be used

facetField - the facet field on which to group our values

max - the max number of values given back (in case of 10 the top 10 will be given)

showTotal - a boolean determining whether the total amount should be given back as the last element of the array

Returns:

an array containing our results

Throws:

org.apache.solr.client.solrj.SolrServerException - ...

queryFacetDate

public ObjectCount[] queryFacetDate(String query,
                           String filterQuery,
                           int max,
                           String dateType,
                           String dateStart,
                           String dateEnd,
                           boolean showTotal,
                           Context context)
                             throws org.apache.solr.client.solrj.SolrServerException

Description copied from interface: SolrLoggerService

Query used to get values grouped by the date.

Specified by:

queryFacetDate in interface SolrLoggerService

Parameters:

query - the query to be used

max - the max number of values given back (in case of 10 the top 10 will be given)

dateType - the type to be used (example: DAY, MONTH, YEAR)

dateStart - the start date Format:(-3, -2, ..) the date is calculated relatively on today

dateEnd - the end date stop Format (-2, +1, ..) the date is calculated relatively on today

showTotal - a boolean determining whether the total amount should be given back as the last element of the array

Returns:

and array containing our results

Throws:

org.apache.solr.client.solrj.SolrServerException - ...

queryFacetQuery

public Map<String,Integer> queryFacetQuery(String query,
                                  String filterQuery,
                                  List<String> facetQueries)
                                    throws org.apache.solr.client.solrj.SolrServerException

Specified by:

queryFacetQuery in interface SolrLoggerService

Throws:

org.apache.solr.client.solrj.SolrServerException

queryTotal

public ObjectCount queryTotal(String query,
                     String filterQuery)
                       throws org.apache.solr.client.solrj.SolrServerException

Specified by:

queryTotal in interface SolrLoggerService

Throws:

org.apache.solr.client.solrj.SolrServerException

getDateView

protected String getDateView(String name,
                 String type,
                 Context context)

query

public org.apache.solr.client.solrj.response.QueryResponse query(String query,
                                                        String filterQuery,
                                                        String facetField,
                                                        int rows,
                                                        int max,
                                                        String dateType,
                                                        String dateStart,
                                                        String dateEnd,
                                                        List<String> facetQueries,
                                                        String sort,
                                                        boolean ascending)
                                                          throws org.apache.solr.client.solrj.SolrServerException

Specified by:

query in interface SolrLoggerService

Throws:

org.apache.solr.client.solrj.SolrServerException

getIgnoreSpiderIPs

public String getIgnoreSpiderIPs()

Description copied from interface: SolrLoggerService

Returns in a filterQuery string all the ip addresses that should be ignored

Specified by:

getIgnoreSpiderIPs in interface SolrLoggerService

Returns:

a string query with ip addresses

optimizeSOLR

public void optimizeSOLR()

Description copied from interface: SolrLoggerService

Maintenance to keep a SOLR index efficient. Note: This might take a long time.

Specified by:

optimizeSOLR in interface SolrLoggerService

shardSolrIndex

public void shardSolrIndex()
                    throws IOException,
                           org.apache.solr.client.solrj.SolrServerException

Specified by:

shardSolrIndex in interface SolrLoggerService

Throws:

IOException

org.apache.solr.client.solrj.SolrServerException

createCore

protected org.apache.solr.client.solrj.impl.HttpSolrServer createCore(org.apache.solr.client.solrj.impl.HttpSolrServer solr,
                                                          String coreName)
                                                               throws IOException,
                                                                      org.apache.solr.client.solrj.SolrServerException

Throws:

IOException

org.apache.solr.client.solrj.SolrServerException

reindexBitstreamHits

public void reindexBitstreamHits(boolean removeDeletedBitstreams)
                          throws Exception

Specified by:

reindexBitstreamHits in interface SolrLoggerService

Throws:

Exception

exportHits

public void exportHits()
                throws Exception

Description copied from interface: SolrLoggerService

Export all SOLR usage statistics for viewing/downloading content to a flat text file. The file goes to a series

Specified by:

exportHits in interface SolrLoggerService

Throws:

Exception

addDocumentsToFile

protected void addDocumentsToFile(Context context,
                      org.apache.solr.common.SolrDocumentList docs,
                      File exportOutput)
                           throws SQLException,
                                  ParseException,
                                  IOException

Throws:

SQLException

ParseException

IOException

generateURL

protected String generateURL(String baseURL,
                 Map<String,String> parameters)
                      throws UnsupportedEncodingException

Throws:

UnsupportedEncodingException

addAdditionalSolrYearCores

protected void addAdditionalSolrYearCores(org.apache.solr.client.solrj.SolrQuery solrQuery)