Package pl.allegro.tech.hermes.consumers.consumer.batch

Class DirectBufferPool

java.lang.Object
pl.allegro.tech.hermes.consumers.consumer.batch.DirectBufferPool
public final class DirectBufferPool extends Object
A pool of ByteBuffers kept under a given memory limit. This class is fairly specific to the needs of the producer. In particular it has the following properties:
  1. There is a special "poolable size" and buffers of this size are kept in a free list and recycled
  2. It is fair. That is all memory is given to the longest waiting thread until it has sufficient memory. This prevents starvation or deadlock when a thread asks for a large chunk of memory and needs to block until multiple buffers are deallocated.

  • Constructor Details

    • DirectBufferPool

      public DirectBufferPool(long memory, int poolableSize, boolean blockOnExhaustion)
      Create a new buffer pool
      Parameters:
      memory - The maximum amount of memory that this buffer pool can allocate
      poolableSize - The buffer size to cache in the free list rather than deallocating
      blockOnExhaustion - This controls the behavior when the buffer pool is out of memory. If true the allocate(int) call will block and wait for memory to be returned to the pool. If false allocate(int) will throw an exception if the buffer is out of memory.

  • Method Details

    • allocate

      public ByteBuffer allocate(int size) throws InterruptedException
      Allocate a buffer of the given size. This method blocks if there is not enough memory and the buffer pool is configured with blocking mode.
      Parameters:
      size - The buffer size to allocate in bytes
      Returns:
      The buffer
      Throws:
      InterruptedException - If the thread is interrupted while blocked
      IllegalArgumentException - if size is larger than the total memory controlled by the pool (and hence we would block forever)
      BufferOverflowException - if the pool is in non-blocking mode and size exceeds the free memory in the pool

    • deallocate

      public void deallocate(ByteBuffer buffer, int size)
      Return buffers to the pool. If they are of the poolable size add them to the free list, otherwise just mark the memory as free.
      Parameters:
      buffer - The buffer to return
      size - The size of the buffer to mark as deallocated, note that this maybe smaller than buffer.capacity since the buffer may re-allocate itself during in-place compression

    • deallocate

      public void deallocate(ByteBuffer buffer)

    • availableMemory

      public long availableMemory()
      the total free memory both unallocated and in the free list

    • unallocatedMemory

      public long unallocatedMemory()
      Get the unallocated memory (not in the free list or in use)

    • queued

      public int queued()
      The number of threads blocked waiting on memory

    • poolableSize

      public int poolableSize()
      The buffer size that will be retained in the free list after use

    • totalMemory

      public long totalMemory()
      The total memory managed by this pool