Package org.odpi.openmetadata.frameworks.connectors.properties

Class PagingIterator

java.lang.Object
org.odpi.openmetadata.frameworks.connectors.properties.beans.PropertyBase
org.odpi.openmetadata.frameworks.connectors.properties.PagingIterator
All Implemented Interfaces:
Serializable, Iterator<ElementBase>
public class PagingIterator extends PropertyBase implements Iterator<ElementBase>
PagingIterator supports an iterator over a list of objects that extend ElementBase. Callers can use it to step through the list just once. If they want to parse the list again, they could use the copy/clone constructor to create a new iterator. PagingIterator provides paging support to enable the list in the iterator to be much bigger than can be stored in memory. It maintains pointers for the full list, and a potentially smaller cached list that is used to batch up elements being retrieved from the property (metadata) server. Thus is combines the ability to process very large lists whilst minimising the chatter with the property server. The class uses a number of pointers and counts. Imagine a list of 4000 columns in a table. We can not retrieve all 4000 columns on one call. So we retrieve subsets over a number of REST calls as the caller walks through the full list.
  • totalElement is 4000 - ie how many elements there are in total.
  • maxCacheSize is the maximum elements we can retrieve in one call
  • cachedElementList.size() is how many elements we have in memory at this time.
cachedElementList.size() is set to maxCacheSize() for all retrieves except for processing the last cache of elements. For example, if the totalElement is 25 and the maxCacheSize() is 10 then we would retrieve 3 caches - the first two would have 10 elements in them and the third will have 5 elements. In the first 2 retrieves, maxCacheSize and cachedElementList.size() are set to 10. In the last one, maxCacheSize==10 and cachedElementList.size()==5.
See Also:

  • Field Details

    • maxCacheSize

      protected int maxCacheSize

    • totalElementCount

      protected int totalElementCount

    • cachedElementStart

      protected int cachedElementStart

    • cachedElementList

      protected List<ElementBase> cachedElementList

    • cachedElementPointer

      protected int cachedElementPointer

    • iterator

      protected PropertyIteratorBase iterator

    • log

      protected static final org.slf4j.Logger log

  • Constructor Details

    • PagingIterator

      public PagingIterator(PropertyIteratorBase iterator, int totalElementCount, int maxCacheSize)
      Typical Constructor creates an iterator with the supplied list of comments.
      Parameters:
      iterator - type-specific iterator that wraps this paging iterator.
      totalElementCount - the total number of elements to process. A negative value is converted to 0.
      maxCacheSize - maximum number of elements that should be retrieved from the property server and cached in the element list at any one time. If a number less than one is supplied, 1 is used.

    • PagingIterator

      public PagingIterator(PropertyIteratorBase iterator, PagingIterator templateIterator)
      Copy/clone constructor. Used to reset iterator element pointer to 0;
      Parameters:
      iterator - type-specific iterator that wraps this paging iterator.
      templateIterator - template to copy; null to create an empty iterator

  • Method Details

    • hasNext

      public boolean hasNext()
      The iterator can only be used once to step through the elements. This method returns a boolean to indicate if it has got to the end of the list yet.
      Specified by:
      hasNext in interface Iterator<ElementBase>
      Returns:
      boolean indicating whether there are more elements.

    • next

      public ElementBase next()
      Return the next element in the list
      Specified by:
      next in interface Iterator<ElementBase>
      Returns:
      ElementBase next element.
      Throws:
      OCFRuntimeException - if there are no more elements in the list or there are problems retrieving elements from the property (metadata) server.

    • getElementCount

      public int getElementCount()
      Return the number of elements in the list.
      Returns:
      elementCount

    • remove

      public void remove()
      Remove the current element in the iterator. This call is not supported and results in an exception
      Specified by:
      remove in interface Iterator<ElementBase>

    • toString

      public String toString()
      Standard toString method.
      Overrides:
      toString in class Object
      Returns:
      print out of variables in a JSON-style