org.wikidata.query.rdf.blazegraph.throttling

Class ThrottlingFilter

java.lang.Object

org.wikidata.query.rdf.blazegraph.throttling.ThrottlingFilter

All Implemented Interfaces:

javax.servlet.Filter, ThrottlingMXBean

public class ThrottlingFilter
extends Object
implements javax.servlet.Filter, ThrottlingMXBean

A Servlet Filter that applies throttling. The throttling is based on the request time consumed and the number of errors. The rational is:

request time

This is a good proxy for how much resource (CPU, IO) are consumed.

errors

A client always in error indicates a problem client side, which should be fixed client side.

Resource consumption is based on token buckets as implemented by bbeck. A token bucket is defined by:

capacity

the maximum number of tokes in the bucket

refill amount

the number of tokens to add to the bucket when refilling

refill period

how often to refill the bucket

This filter has two buckets, one to keep track of time, and one to keep track of errors. Each time an error occurs, a token is taken out of the error bucket. Each refill period, tokens are added again. The time bucket has a similar behaviour. As an optimization, we start keeping track of resource consumption only if:

1. a request is taking a significant time
2. a request is in error

The client is throttled if either the time bucket or the error bucket is empty. Since we don't know in advance the cost of a request or if it is going to be in error, the throttling will only occur for the next requests. In case of throttling, the client is notified by an HTTP 429 status code and is presented with a Retry-After HTTP header giving a backoff time in seconds. The clients are segmented in different buckets and resource consumption is tracked individually for each of those buckets. The segmentation is done by [IP address, User Agent], but could be extended to support more complex strategies. A bucket is only kept while its client is active. After a period of inactivity, the bucket is deleted. All state is limited to a single JVM, this filter is not cluster aware.

Constructor Summary

Constructors

Constructor and Description
ThrottlingFilter()

Method Summary

Modifier and Type	Method and Description
static Callable<ThrottlingState>	createThrottlingState(int timeBucketCapacityInSeconds, int timeBucketRefillAmountInSeconds, int timeBucketRefillPeriodInMinutes, int errorBucketCapacity, int errorBucketRefillAmount, int errorBucketRefillPeriodInMinutes) Create Callable to initialize throttling state.
void	destroy() Unregister MBean.
void	doFilter(javax.servlet.ServletRequest request, javax.servlet.ServletResponse response, javax.servlet.FilterChain chain) Check resource consumption and throttle requests as needed.
long	getNumberOfThrottledRequests() Monotonic increasing counter of the the number of requests that have been throttled.
long	getStateSize() The number of entries in the state store.
void	init(javax.servlet.FilterConfig filterConfig) Initialise the filter.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ThrottlingFilter

public ThrottlingFilter()

Method Detail

init

public void init(javax.servlet.FilterConfig filterConfig)
          throws javax.servlet.ServletException

Initialise the filter. See loadStringParam(String, FilterConfig) for the details of where the configuration is loaded from.

Specified by:

init in interface javax.servlet.Filter

Parameters:

filterConfig -

Throws:

javax.servlet.ServletException

createThrottlingState

public static Callable<ThrottlingState> createThrottlingState(int timeBucketCapacityInSeconds,
                                                              int timeBucketRefillAmountInSeconds,
                                                              int timeBucketRefillPeriodInMinutes,
                                                              int errorBucketCapacity,
                                                              int errorBucketRefillAmount,
                                                              int errorBucketRefillPeriodInMinutes)

Create Callable to initialize throttling state.

Parameters:

timeBucketCapacityInSeconds -

timeBucketRefillAmountInSeconds -

timeBucketRefillPeriodInMinutes -

errorBucketCapacity -

errorBucketRefillAmount -

errorBucketRefillPeriodInMinutes -

doFilter

public void doFilter(javax.servlet.ServletRequest request,
                     javax.servlet.ServletResponse response,
                     javax.servlet.FilterChain chain)
              throws IOException,
                     javax.servlet.ServletException

Check resource consumption and throttle requests as needed.

Specified by:

doFilter in interface javax.servlet.Filter

Parameters:

request -

response -

chain -

Throws:

IOException

javax.servlet.ServletException

destroy

public void destroy()

Unregister MBean.

Specified by:

destroy in interface javax.servlet.Filter

getStateSize

public long getStateSize()

The number of entries in the state store. This represents the number of clients currently being tracked.

Specified by:

getStateSize in interface ThrottlingMXBean

getNumberOfThrottledRequests

public long getNumberOfThrottledRequests()

Monotonic increasing counter of the the number of requests that have been throttled.

Specified by:

getNumberOfThrottledRequests in interface ThrottlingMXBean