Incremental Data Retrieval

Problem Outline

Imagine there is a database table containing a large number of objects and a catalog built on this table. For the objects to be displayed in the catalog, they have to be read from the database to the operating memory. Depending on the amount of data and the size of available memory, this may either be impossible at all (causing memory overflow) or it may significantly lower the application's performance.

Solution

Read Limit

In earlier versions, the problem was dealt with by setting a limit on the maximum number of objects read to memory (Read Limit).

• If the number of objects (matching a search condition, if existent) in the database was smaller or equal to the Read Limit, all objects were read, sorted in memory and displayed.

• If the number of objects to be read exceeded the Read Limit, only as many objects as specified by the Read Limit were read, sorted in memory and displayed. A warning message informed the user that there were more objects than the displayed ones in the database. To access all objects matching the search condition, the condition had to be made more restrictive.

Incremental Data Retrieval

This currently used method is based on sorting the objects (matching a search condition, if existent) directly in the database, then reading just a limited "reasonable" number of them, with more objects read dynamically on an on-demand basis. See example.

Incremental Data Retrieval is used when reading the following selections of objects:

• Selection resulting from a search condition

  E.g. opening a catalog from the sidebar and performing searches in the catalog

• Selection resulting from a relationship between objects

  E.g. using a reference list or opening a sub-node in a tree view of a catalog