org.dspace.statistics.util

Class SpiderDetector

java.lang.Object

org.dspace.statistics.util.SpiderDetector

public class SpiderDetector
extends java.lang.Object

SpiderDetector is used to find IP's that are spiders... In future someone may add Host Domains to the detection criteria here.

Author:

kevinvandevelde at atmire.com, ben at atmire.com, Mark Diggory (mdiggory at atmire.com)

Constructor Summary

Constructors

Constructor and Description
SpiderDetector()

Method Summary

Methods

Modifier and Type	Method and Description
static java.util.Set<java.lang.String>	getSpiderIpAddresses() Get an immutable Set representing all the Spider Addresses here
static boolean	isSpider(javax.servlet.http.HttpServletRequest request) Static Service Method for testing spiders against existing spider files.
static boolean	isSpider(java.lang.String ip) Check individual IP is a spider.
static boolean	isSpider(java.lang.String clientIP, java.lang.String proxyIPs, java.lang.String hostname, java.lang.String agent) Static Service Method for testing spiders against existing spider files.
static java.util.Set<java.lang.String>	readPatterns(java.io.File patternFile) Utility method which reads lines from a file & returns them in a Set.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SpiderDetector

public SpiderDetector()

Method Detail

readPatterns

public static java.util.Set<java.lang.String> readPatterns(java.io.File patternFile)
                                                    throws java.io.IOException

Utility method which reads lines from a file & returns them in a Set.

Parameters:

patternFile - the location of our spider file

Returns:

a vector full of patterns

Throws:

java.io.IOException - could not happen since we check the file be4 we use it

getSpiderIpAddresses

public static java.util.Set<java.lang.String> getSpiderIpAddresses()

Get an immutable Set representing all the Spider Addresses here

Returns:

isSpider

public static boolean isSpider(java.lang.String clientIP,
               java.lang.String proxyIPs,
               java.lang.String hostname,
               java.lang.String agent)

Static Service Method for testing spiders against existing spider files.

In future spiders HashSet may be optimized as byte offset array to improve performance and memory footprint further.

Parameters:

clientIP - address of the client.

proxyIPs - comma-list of X-Forwarded-For addresses, or null.

hostname - domain name of host, or null.

agent - User-Agent header value, or null.

Returns:

true if the client matches any spider characteristics list.

isSpider

public static boolean isSpider(javax.servlet.http.HttpServletRequest request)

Static Service Method for testing spiders against existing spider files.

Parameters:

request -

Returns:

true|false if the request was detected to be from a spider.

isSpider

public static boolean isSpider(java.lang.String ip)

Check individual IP is a spider.

Parameters:

ip -

Returns:

if is spider IP