org.wikidata.query.rdf.blazegraph.throttling

Class ThrottlingFilter

java.lang.Object

org.wikidata.query.rdf.blazegraph.throttling.ThrottlingFilter

All Implemented Interfaces:

javax.servlet.Filter, ThrottlingMXBean

public class ThrottlingFilter
extends Object
implements javax.servlet.Filter, ThrottlingMXBean

A Servlet Filter that applies throttling. The throttling is based on the request time consumed and the number of errors. The rational is:

request time

This is a good proxy for how much resource (CPU, IO) are consumed.

errors

A client always in error indicates a problem client side, which should be fixed client side.

Resource consumption is based on token buckets as implemented by bbeck. A token bucket is defined by:

capacity

the maximum number of tokes in the bucket

refill amount

the number of tokens to add to the bucket when refilling

refill period

how often to refill the bucket

This filter has two buckets, one to keep track of time, and one to keep track of errors. Each time an error occurs, a token is taken out of the error bucket. Each refill period, tokens are added again. The time bucket has a similar behaviour. As an optimization, we start keeping track of resource consumption only if:

1. a request is taking a significant time
2. a request is in error

The client is throttled if either the time bucket or the error bucket is empty. Since we don't know in advance the cost of a request or if it is going to be in error, the throttling will only occur for the next requests. In case of throttling, the client is notified by an HTTP 429 status code and is presented with a Retry-After HTTP header giving a backoff time in seconds. Further more, if a client does not back off when being sent HTTP 429, and it looks abusive, it is completely banned for a period of time, and will receive HTTP 403 (Forbidden) during the duration of the ban. The clients are segmented in different buckets and resource consumption is tracked individually for each of those buckets. The segmentation is done by [IP address, User Agent], but could be extended to support more complex strategies. A bucket is only kept while its client is active. After a period of inactivity, the bucket is deleted. All state is limited to a single JVM, this filter is not cluster aware.

Constructor Summary

Constructors

Constructor and Description
ThrottlingFilter()

Method Summary

Modifier and Type	Method and Description
static Callable<ThrottlingState>	createThrottlingState(Duration timeBucketCapacity, Duration timeBucketRefillAmount, Duration timeBucketRefillPeriod, int errorBucketCapacity, int errorBucketRefillAmount, Duration errorBucketRefillPeriod, int throttleBucketCapacity, int throttleBucketRefillAmount, Duration throttleBucketRefillPeriod, Duration banDuration) Create Callable to initialize throttling state.
void	destroy() Unregister MBean.
void	doFilter(javax.servlet.ServletRequest request, javax.servlet.ServletResponse response, javax.servlet.FilterChain chain) Check resource consumption and throttle requests as needed.
long	getNumberOfBannedRequests() Monotonic increasing counter of the the number of requests that have been banned.
long	getNumberOfThrottledRequests() Monotonic increasing counter of the the number of requests that have been throttled.
long	getStateSize() The number of entries in the state store.
void	init(javax.servlet.FilterConfig filterConfig) Initialise the filter.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ThrottlingFilter

public ThrottlingFilter()

Method Detail

init

public void init(javax.servlet.FilterConfig filterConfig)

Initialise the filter. The following parameters are available (see TokenBucket for the details on bucket configuration, see implementation for the default values):

• request-duration-threshold-in-millis: requests longer than this threshold will start the tracking for this user
• time-bucket-capacity-in-seconds, time-bucket-refill-amount-in-seconds, time-bucket-refill-period-in-minutes: configuration of the bucket tracking request durations
• error-bucket-capacity, error-bucket-refill-amount, error-bucket-refill-period-in-minutes: configuration of the bucket tracking errors
• throttle-bucket-capacity, throttle-bucket-refill-amount, throttle-bucket-refill-period-in-minutes: configuration of the bucket tracking throttling
• ban-duration-in-minutes: how long should a user be banned when a ban is triggered
• max-state-size: how many users to track
• state-expiration-in-minutes: tracking of a user expires after this duration
• enable-throttling-if-header: enable the throttling on the requests which have this header set
• enable-ban-if-header: enable the banning on the requests which have this header set
• always-throttle-param: always throttle requests where this parameter is set (useful for testing)
• always-ban-param: always ban requests where this parameter is set (useful for testing)
• enabled: entirely disable this filter if set to false

See ThrottlingFilterConfig.loadStringParam(String, FilterConfig) for the details of where the configuration is loaded from.

Specified by:

init in interface javax.servlet.Filter

Parameters:

filterConfig -

createThrottlingState

public static Callable<ThrottlingState> createThrottlingState(Duration timeBucketCapacity,
                                                              Duration timeBucketRefillAmount,
                                                              Duration timeBucketRefillPeriod,
                                                              int errorBucketCapacity,
                                                              int errorBucketRefillAmount,
                                                              Duration errorBucketRefillPeriod,
                                                              int throttleBucketCapacity,
                                                              int throttleBucketRefillAmount,
                                                              Duration throttleBucketRefillPeriod,
                                                              Duration banDuration)

Create Callable to initialize throttling state.

doFilter

public void doFilter(javax.servlet.ServletRequest request,
                     javax.servlet.ServletResponse response,
                     javax.servlet.FilterChain chain)
              throws IOException,
                     javax.servlet.ServletException

Check resource consumption and throttle requests as needed.

Specified by:

doFilter in interface javax.servlet.Filter

Parameters:

request -

response -

chain -

Throws:

IOException

javax.servlet.ServletException

destroy

public void destroy()

Unregister MBean.

Specified by:

destroy in interface javax.servlet.Filter

getStateSize

public long getStateSize()

Description copied from interface: ThrottlingMXBean

The number of entries in the state store. This represents the number of clients currently being tracked.

Specified by:

getStateSize in interface ThrottlingMXBean

getNumberOfThrottledRequests

public long getNumberOfThrottledRequests()

Description copied from interface: ThrottlingMXBean

Monotonic increasing counter of the the number of requests that have been throttled.

Specified by:

getNumberOfThrottledRequests in interface ThrottlingMXBean

getNumberOfBannedRequests

public long getNumberOfBannedRequests()

Description copied from interface: ThrottlingMXBean

Monotonic increasing counter of the the number of requests that have been banned.

Specified by:

getNumberOfBannedRequests in interface ThrottlingMXBean