to.etc.dbpool

Class ConnectionPool

java.lang.Object

to.etc.dbpool.ConnectionPool

public final class ConnectionPool
extends Object

Working

This represents an actual pool for a single database. It collects a list of connections to the same database. The connection pool is a generic provider for database connections of several types. These can be pooled connections but the thing can also be used to provide connections without a pooled set being built.

To start a pool it needs to be defined first. Defining the pool means that the definePool() call in the PoolManager has been called, providing the pool's data like it's ID, the database driver to use, the userid/password etc etc. Defining a pool does not allocate a pool of connections. After definition all of the connection calls work but when a connection is closed it gets discarded as well, i.e. it is not returned to the pool.

To start pooling connections you need to call initializePool() after it's definition. This forces the pool to allocate it's minimum number of available connections, and new connections will be retrieved from here. When connections are closed they will be returned to the pool unless the max #of open connections is reached in which case the connection will be discarded.

Connections from the pool

Connections returned by this code are always wrapped by a Connection wrapper. We call these connections Connection proxies. To obtain the actual connection (for instance when you need to do driver-specific calls) you need to cast the Connection to the PooledConnection wrapper and call getRealConnection() on it.

All of the pools' connections do resource management: all resources (statements, result sets) obtained from the connection is kept in a list with the connection so that they can be released/closed as soon as the connection proxy is closed. This prevents reused real connections from having open statements. It also means that resource leaks are minimized: as long as the connection is closed it also closes all open cursors.

Main connection types

Two main types of connection exist: Pooled connections and Unpooled connections. The latter is a bit of a misnomer but the term is kept because too much code uses these terms.

Pooled connections

Pooled connections are connections that are used in server applications. These are obtained from the set of free connections if possible and are subject to several checks:

• They must be closed again after allocation as soon as possible. If a pooled connection is used too long (> 8 seconds) the system will generate a warning message containing the context of the connection. This mechanism is meant as a server optimization method; calls within the server should never do database processing that takes that long.
• They are forcefully closed if they are used for more than a minute. A Janitor process scans all used connections and if it finds a connection that's in use for more than a minute it closes both the connection's proxy (the thing that is used by the client) and the real connection used. This ensures that any running code on that database is terminated. The client using the connection will get an exception as soon as it tries to use it's proxy again. This check is done only if the pool operates in pooled mode (initializePool() has been called).

Unpooled connections

Unpooled connections are connections that do not have the above checks executed on them. As such they can execute database code that takes longer for instance for deamon-like tasks. If the pool operates in pooled mode the connections are allocated from the available connections in the pool so they will be provided quickly.

In addition, Unpooled connections are not counted as "used" connections from the pool's connection set. This means an unlimited amount of Unpooled connections can be allocated.

Closing connections

By default these connection types are closeable, meaning that calling close() on the connection will discard all resources and return the real connection back to the pool. It is however possible to set a connection proxy into uncloseable mode with a call to setCloseable(false). After this call all calls to close() are silently ignored. To actually close the proxy you need to call closeForced().

This is used in server code where connections are cached during a request. As a server executes many code sections during the handling of a request the overhead of allocating connections for every database action is large. Another problem is that calling code from other code can cause multiple connections to be allocated to the same database, reducing the ability of the pool to provide connections. To prevent this we usually want to "cache" connections once allocated during the scope of a request. This caching can easily be done by setting the connection to uncloseable at allocation time and closing it in a top-level code part just before the request terminates. Setting the connection to uncloseable ensures that if code calls close the connection stays alive for other code.

Thread connections

Besides explicit connection caching where a connection is obtained, set to uncloseable and reused by calls there is another way to cache connections during the lifetime of a request: by using ThreadConnections. A thread connection can be pooled or unpooled as usual. When a thread connection is allocated the connection is registered to belong to the thread which allocated it. This thread becomes the "owner" of that thread. The current ThreadConnection for any given pool and Thread can be quickly obtained by looking in a per-thread hashtable mapped by pool.

The first time a thread allocates a ThreadConnection it will not have a current one so a new connection of the requested base type (pooled, unpooled) is allocated and saved in the thread's map. This connection will then be set to uncloseable to prevent it from being closed inadvertedly.

The next time the thread requests a ThreadConnection this stored copy will be returned, allowing for reuse of a connection during the time it is allocated to the thread.

Since a ThreadConnection is uncloseable calling close() on it will not close the connection; this must be done either by calling closeForced() or by calling the closeThreadConnections() call on the poolmanager. This call will walk the connection list for the calling thread and discard all of the thread connections allocated therein. After this call a new call to allocate a thread connection will allocate a new connection from the poolset.

Interaction between ThreadConnections and the other shit

Unfortunately the "cache-it-myself" and the ThreadConnections method of connection caching do not mix that well. It would be best to select one method for a complete system and not use the other. If this is impossible (because multiple code bases each use their own approach like NEMA) there are some special semantics to remember.

Mixing both methods works best if the "cache-it-yourself" method uses a ThreadConnection and does not forcefully close it, leaving that to the closeThreadConnections() call. If the "cache-it-yourself" method allocates a pooled connection (not a thread connection) and another piece of code allocates a thread connection this will cause a single thread to use two connections (possibly to the same database). This cannot be avoided because forcing the pooled connection to check for a threadconnection first causes a shitload of trouble when that connection gets closed.

Version:

$Version$

Author:

jal

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ConnectionPool.ErrorEntry
static interface	ConnectionPool.IPoolEvent Pool event, add listeners using

Field Summary

Fields

Modifier and Type	Field and Description
static Logger	ALLOC
static Logger	JAN
protected boolean	m_dbg_stacktrace T if this pool has stack tracing enabled.
protected long	m_n_open_rs The #of resultsets opened by all statements in the pool
protected int	m_n_open_stmt The #of statements CURRENTLY allocated by the pool
protected long	m_n_rows Deprecated.
protected int	m_peak_open_stmt The #of statements MAX allocated by the pool
protected long	m_statementTotalPrepareCount The #of prepare statements executed.
static Logger	MSG
static long	STMT_START_MAGIC

Constructor Summary

Constructors

Constructor and Description
ConnectionPool(PoolManager pm, String id, PoolConfig config)

Method Summary

Modifier and Type	Method and Description
void	addListener(ConnectionPool.IPoolEvent l) Add a listener for pool related events.
void	addPlSqlDebugHandler(String plsqldebug)
PoolConfig	c() Return the config parameter class.
boolean	dbgIsStackTraceEnabled() Returns T if stack tracking is enabled for debugging purposes.
void	dbgRelease(String what, Connection dbc)
void	dbgSetStacktrace(boolean on) Switches stack tracing ON.
Object	getAttribute(String name)
IConnectionStatisticsFactory	getConnectionStatisticsFactory()
int	getConnectionUsedTooLongWarningTimeout() Returns the #of seconds that a connection must have been USED before a warning and a stack dump is generated.
int	getForceTimeout()
String	getID()
PoolManager	getManager() Return the owner pool manager.
<T> T	getOrCreateAttribute(String name, java.util.function.Supplier<T> supplier)
DataSource	getPooledDataSource()
PoolStats	getPoolStatistics() Copies all pool data into the poolStats structure.
List<ConnectionPool.ErrorEntry>	getSavedErrorList()
Connection	getUnpooledConnection(String username, String password)
DataSource	getUnpooledDataSource()
List<ConnectionProxy>	getUsedConnections() Get a list of all ConnextionProxy's currently in use.
int[]	getUseTimeTable()
String	getUseTimeTableStr() Returns a HTML-formatted table of connection usage times.
boolean	hasSavedErrors()
void	initialize() Tries to put the pool in "pooled mode".
boolean	isCommitDisabled() Returns T if commits are disabled for the current thread.
boolean	isFileLogging()
boolean	isPooledMode()
boolean	isSavingErrors()
void	removeListener(ConnectionPool.IPoolEvent l) Remove the specified listener.
void	saveError(String subject, String msg)
boolean	scanExpiredConnections(int scanIntervalInSeconds, boolean forcedisconnects) This function gets called from the broker's janitor thread, OR from the purgatory handler (the thing called when all connections are used).
void	setAttribute(String name, Object value)
void	setCommitDisabled(boolean on) This enables or disables commits for connections allocated/used by this thread.
void	setFileLogging(File target)
void	setForceTimeout(int timeout) When set > 0, this will call setTimeout on all statements and calls.
void	setSaveErrors(boolean on)
void	writeSpecial(ConnectionProxy cp, byte stmtType)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MSG

public static final Logger MSG

JAN

public static final Logger JAN

ALLOC

public static final Logger ALLOC

m_n_open_stmt

protected int m_n_open_stmt

The #of statements CURRENTLY allocated by the pool

m_peak_open_stmt

protected int m_peak_open_stmt

The #of statements MAX allocated by the pool

m_n_open_rs

protected long m_n_open_rs

The #of resultsets opened by all statements in the pool

m_statementTotalPrepareCount

protected long m_statementTotalPrepareCount

The #of prepare statements executed.

m_n_rows

@Deprecated
protected long m_n_rows

Deprecated.

m_dbg_stacktrace

protected boolean m_dbg_stacktrace

T if this pool has stack tracing enabled.

STMT_START_MAGIC

public static final long STMT_START_MAGIC

See Also:

Constant Field Values

Constructor Detail

ConnectionPool

public ConnectionPool(PoolManager pm,
                      String id,
                      PoolConfig config)
               throws SQLException

Throws:

SQLException

Method Detail

c

public PoolConfig c()

Return the config parameter class.

Returns:

addListener

public void addListener(@Nonnull
                        ConnectionPool.IPoolEvent l)

Add a listener for pool related events.

Parameters:

l -

removeListener

public void removeListener(@Nonnull
                           ConnectionPool.IPoolEvent l)

Remove the specified listener.

Parameters:

l -

addPlSqlDebugHandler

public void addPlSqlDebugHandler(@Nonnull
                                 String plsqldebug)
                          throws SQLException

Throws:

SQLException

initialize

public void initialize()
                throws SQLException

Tries to put the pool in "pooled mode". If the pool already is in pooled mode we are done before we knew it ;-)

Throws:

SQLException

dbgRelease

public void dbgRelease(String what,
                       Connection dbc)

getUsedConnections

public List<ConnectionProxy> getUsedConnections()

Get a list of all ConnextionProxy's currently in use. It gets that list with "this" locked but locks nothing else. The entries returned are live, so by the time you are using them they can be dead (because they are closed/invalidated at that time).

We return the proxies, not the entries, because the proxies remain valid for a single use context. It means they are stable in time and have a single life cycle (life, dead).

Returns:

getUnpooledConnection

public Connection getUnpooledConnection(String username,
                                        String password)
                                 throws SQLException

Parameters:

username -

password -

Returns:

Throws:

SQLException

scanExpiredConnections

public boolean scanExpiredConnections(int scanIntervalInSeconds,
                                      boolean forcedisconnects)

This function gets called from the broker's janitor thread, OR from the purgatory handler (the thing called when all connections are used).

getConnectionUsedTooLongWarningTimeout

public int getConnectionUsedTooLongWarningTimeout()

Returns the #of seconds that a connection must have been USED before a warning and a stack dump is generated.

Returns:

getUseTimeTable

public int[] getUseTimeTable()

getUseTimeTableStr

public String getUseTimeTableStr()

Returns a HTML-formatted table of connection usage times.

Returns:

setSaveErrors

public void setSaveErrors(boolean on)

hasSavedErrors

public boolean hasSavedErrors()

isSavingErrors

public boolean isSavingErrors()

getSavedErrorList

public List<ConnectionPool.ErrorEntry> getSavedErrorList()

saveError

public void saveError(String subject,
                      String msg)

isFileLogging

public boolean isFileLogging()

setFileLogging

public void setFileLogging(File target)

writeSpecial

public void writeSpecial(ConnectionProxy cp,
                         byte stmtType)

getID

public String getID()

getManager

public final PoolManager getManager()

Return the owner pool manager.

Returns:

isPooledMode

public boolean isPooledMode()

getUnpooledDataSource

public DataSource getUnpooledDataSource()

getPooledDataSource

public DataSource getPooledDataSource()

setForceTimeout

public void setForceTimeout(int timeout)

When set > 0, this will call setTimeout on all statements and calls.

Parameters:

timeout -

getForceTimeout

public int getForceTimeout()

dbgIsStackTraceEnabled

public boolean dbgIsStackTraceEnabled()

Returns T if stack tracking is enabled for debugging purposes.

dbgSetStacktrace

public void dbgSetStacktrace(boolean on)

Switches stack tracing ON. This is very expensive and should only be used in case of trouble. Switching on stack trace (done thru the /nema/ servlet path) causes the pool to remember the last 10 stack paths that accessed a connection.

setCommitDisabled

public void setCommitDisabled(boolean on)

This enables or disables commits for connections allocated/used by this thread. Primary use is for JUnit tests, preventing them from changing the database.

Parameters:

on -

Since:

2011/08/12

isCommitDisabled

public boolean isCommitDisabled()

Returns T if commits are disabled for the current thread. This can be used to prevent JUnit tests from changing the database.

Returns:

Since:

2011/08/12

getPoolStatistics

public PoolStats getPoolStatistics()

Copies all pool data into the poolStats structure. This provides a point-in-time copy of all pool statistics in proper relation.

setAttribute

public void setAttribute(@Nonnull
                         String name,
                         @Nullable
                         Object value)

getAttribute

public Object getAttribute(@Nonnull
                           String name)

getOrCreateAttribute

public <T> T getOrCreateAttribute(@Nonnull
                                  String name,
                                  @Nonnull
                                  java.util.function.Supplier<T> supplier)

getConnectionStatisticsFactory

@Nullable
public IConnectionStatisticsFactory getConnectionStatisticsFactory()