Paged Search on Resources

Last modified 14 Jan 2025 21:57 +01:00
EXPERIMENTAL
This feature is experimental. It means that it is not intended for production use. The feature is not finished. It is not stable. The implementation may contain bugs, the configuration may change at any moment without any warning and it may not work at all. Use at your own risk. This feature is not covered by midPoint support. In case that you are interested in supporting development of this feature, please consider purchasing midPoint Platform subscription.
OUTDATED
This page is outdated, it contains information that was not updated in a long time. The described functionality may or may not work. Do not rely on information provided on this page.

There is a feature of the ConnId framework since version 1.4: paged search. The client can specify that it would like to retrieve not all query results in one operation, but only a part of it - specified by an offset and an item count.

By default, midPoint does not utilize the paged search when retrieving accounts from a resource. For background activities, like live sync and reconciliation, this is not necessary at all. However, for listing accounts (Resources → All resources → select a resource → Resource objects) such a feature is really useful, if you have many accounts on your resource. In non-paged case, midPoint has to retrieve all accounts up to the currently displayed page (and it does not know how many accounts are there, so showing something like "Displaying 1 to 20, unknown number of matching results."). When using paged search, it is sufficient to retrieve only those records, which are to be displayed. And midPoint knows how many objects are there.

How to enable paged search?

First of all, you have to use a connector that supports paged searches.

Now you have to specify that paged searches are to be used. You can do this by creating <pagedSearch> capability - either at the level of resource or a resource object type. A minimal config looks like this:

At the level of resource:

<capabilities>
    <native> ... </native>
    <configured xmlns:cap="http://midpoint.evolveum.com/xml/ns/public/resource/capabilities-3">
        <cap:pagedSearch/>
    </configured>
</capabilities>

At the level of resource object type:

<schemaHandling>
    <objectType>
        <kind>account</kind>
        <displayName>Default Account</displayName>
        <default>true</default>
        ...
        <configuredCapabilities xmlns:cap="http://midpoint.evolveum.com/xml/ns/public/resource/capabilities-3">
            <cap:pagedSearch/>
        </configuredCapabilities>
        ...

As paged searches require sorting of any kind (without sorting, the concept of a "page" has no meaning), you may define a default sorting - a field and an order. If you do not specify it, the connector may - or may not - supply its own default sorting. E.g. AD connector uses Common Name as a sorting criteria as the default. But you can use e.g. "sn", as indicated below:

<configuredCapabilities xmlns:cap="http://midpoint.evolveum.com/xml/ns/public/resource/capabilities-3">
    <cap:pagedSearch>
        <cap:defaultSortField xmlns:ri="http://midpoint.evolveum.com/xml/ns/public/resource/instance-3">ri:sn</cap:defaultSortField>
    </cap:pagedSearch>
 </configuredCapabilities>

Finally, you have to tell midPoint to use paged search to estimate the number of objects. To do that, countObjects capability has to be used, e.g.

<configuredCapabilities xmlns:cap="http://midpoint.evolveum.com/xml/ns/public/resource/capabilities-3">
    <cap:pagedSearch/>
    <cap:countObjects>
        <cap:simulate>pagedSearchEstimate</cap:simulate>
    </cap:countObjects>
</configuredCapabilities>

Note that there is also a possibility to set simulate to "sequentialSearch". This method does not require paged searches. However, it is quite inefficient, because

  1. it causes midPoint to fetch all resource objects in order to know their count;

  2. when displaying objects in the middle of the list (e.g. those from 1001 to 1010), midPoint has to uselessly fetch first 1000 objects, just to get the required ones. Paged search is much more effective than this.

  1. AD/Exchange connector implements paged search for objects that get retrieved from AD: users, orgs, groups. It does not support this kind of search for objects retrieved using PowerShell (all other Exchange-related object classes).

  2. The sorting key must be indexed in AD. For example, icfs:name (a.k.a. __NAME__ a.k.a. distinguishedName) is not, neither is ad_container attribute. Among attributes that are indexed by default are cn and sn. The cn attribute is used by the connector if no sorting key is specified.

  3. The implementation is based on VLV. So, in order to work, the VLV must not be disabled (see e.g. here).

  4. If you still get the "The server does not support the requested critical extension." error, consider

    1. applying kb977180-v2

    2. setting HKLM\System\CurrentControlSet\Services\NTDS\Parameters\DSA Heuristics to 000000000001 (string) - see this discussion thread - although beware if this fits into your environment!

  5. If you encounter problems with queries with large result sets, you might want to tune your server’s LDAP parameters.

Was this page helpful?
YES NO
Thanks for your feedback