Query API

Last modified 12 Mar 2022 00:22 +01:00
Table of Contents
This document is up-to-date for midPoint 4.4 LTS version.

Objects, Queries and Filters

Prism and Resource objects

MidPoint works with data structures called Prism objects. These are used for midPoint internal data, but they can also represent remote objects stored on the Resources, e.g. accounts and groups. Prism objects are stored in the Repository to preserve long-term midPoint state. Collections of Prism objects are worked with frequently too. Often we need to extract only objects matching some specfic criteria, either from a runtime collection or from the repository. For that we use filters and the Query API described in this document.

Each Prism object is conceptually a tree. Tree nodes are called prism items: containers, references and properties. Each item has one or more values: container values, reference values or property values. See Prism Data Structures for more. Some items are strictly single-valued, others may contain multiple values.

The resource objects are represented as Shadow objects inside midPoint. Remote attributes of interests are stored in a single-valued container called attributes. You can either query Shadow objects, which is a query against the repository, or you can query remote resource objects. But resources generally don’t understand midPoint Query language, and the query must be translated. For instance, when a ConnId Connector is used, the Query is translated for the ConnId framework. Limitations are mentioned in this document later.

Query

Query language is defined in Prism project in query-3 schema.

Object and Container value query

A query can be:

  • An object query, targeted at one or more object classes, for example:

    • "give me all users with a given name",

    • "give me all the objects with a given name",

    • "give me all focal objects with a given assignment", etc.

  • A container value query, often called just "container query".

    Such queries are targeted at container values. These are being introduced in the context of special objects, like certification campaigns and lookup tables, or even in cases when no parent object exists, like audit events. Examples of queries:

    • "give me all certification cases related to this reviewer, irrespective to in which campaign they are contained",

    • "give me all rows for this particular lookup table that start with this prefix",

    • "give me all work items allocated to particular user".

Container value queries were introduced for the following reasons:

  • Querying the objects would still return all the container values, even non-matching. Container value query only returns the values we need, with their parent objects as necessary. Simply put, object query filters based on container values, but does not filter the container values. With the object query we would still need to filter the container values in memory somehow.

  • While not the same as the previous point, but often related - performance. Sometimes we need a dozen of container values from a few objects, while these objects have many thousands of such container values in total. We don’t want these values, we don’t want to load them from the repository and process them to filter them out.

  • Also, some containers are standalone and have no parent objects ("owner"). This is the case of audit records.

Query anatomy

Each query consists of two parts:

  • Filter specifying which records (objects, container values) are to be retrieved using various conditions. Filter can be primitive or complex, we will talk about various filter types in the following sections.

  • Paging instruction saying how to sort the results and what range, or page, should be obtained, e.g. those with numbers 100-199.

Both parts are optional and the query can actually be empty.

Few more pieces of information are needed for successful query execution. These are typically delivered together with the query, but are not part of it:

  • Type of the object or container value we want to query.

  • Additional options fine-tuning the search and retrieval process. Options can specify what optional pieces to fetch and also wheter DISTINCT should be applied to the results.

This additional information is either implied or can be provided close to the query as needed.

Query playground

To experiment with the query language, there is hardly a better place than the actual running midPoint. Log in the GUI as administrator and chose Query playground in the main menu on the left, all the way down, just above About.

You have to:

  • Select the Object type,

  • check Distinct if needed (first try without it),

  • and finally write the query into the text area.

Alternatively use an existing example from the selection box below.

You can either execute the query or just simulate it. In both cases you will see the translated SQL query (or HQL for the old Generic repository) and the parameter values.

The distinct option is often essential to get the right count of objects when searching in the Generic repository. This is caused by LEFT JOIN used even when traversing to multi-value containers. E.g., filter on assignment/targetRef causes each object with multiple assignments to be returned multiple times. Using Exists filter (see later) does not fix this in the Generic repository either, as it also uses LEFT JOIN internally.

The new Native repository does not suffer by these problems as it always uses SQL EXISTS when traversing to multi-value containers and also for Exists filter. Native repository actually ignores distinct if there is no JOIN in the final query, as the returned raws must be distinct. Native repository uses LEFT JOIN only to traverse across single-valued containers and references and their targets, so even then the distinct option is useless and, when honoured, can potetinally hurt SQL performance.

Primitive filters

Primitive filters are these that do not contain other filters. They are of two categories:

Trivial filters

Filter Description

None filter

Passes no values, i.e. always evaluates to "false".

All filter

Passes all values, i.e. always evaluates to "true".

Undefined filter

Treated like nonexistent or invisible filter. For all fiters F1 and F2 the following holds:

  • F1 && Undefined = F1

  • F2 || Undefined = F2

Value filters

These filters decide on value(s) of a given property, reference or container.

Generally, they are characterized by:

  • A left-side item path, pointing to a property or a reference. The item can be single-value or multi-value. There are generally no surprises for single-value items. Multi-value items can have various limitations for some operations depending on the query engine (provisioning, repository…​).

  • A right-side constant value(s) or item path, used as the other operand for the filter operation. Item path on the right side has a limited support only for repository engine.

  • Optionally, a matching rule.

Filter Applicable left-side items for repo queries Applicable left-side items for resource queries Applicable right-side constant values Applicable right-side path-pointed values Description

Equal filter

property

property

null, single-value, limited multi-value support (see description)

limited support for repository: single-valued property

For null filter value: Accepts if property has no values, e.g. IS NULL for SQL repository.

For single filter value: Accepts if one of the left-hand property values is the same as filter value.

  • If left-hand item is a single-value property, it is simple equals semantics.

  • If left-hand item is a multi-value property, it is any equals semantics.

  • If left-hand item is single-value property in multi-value container, it is any equals semantis.

For multiple filter values: Accepts if one of the left-hand property values is the same as any of the filter values.

  • If left-hand item is a single-value property, this is equivalent to SQL IN operation.

  • If left-hand item is a multi-value property, it means any in semantics when repository and in-memory query engine is used. For the resource query, the filter accepts if the attribute contains all the filter values.

Combinations with multiple filter values have limited support with the new Native repository. There is no official support for this when using the old Generic repository. See the section after this table.

Resource and in-memory queries do not support items on the right side of an operator. Only constant values may be present there.

Greater, Less filter

property, limited multi-value support

property

single, non-null

singleton

Accepts if one of property values is greater/greater-or-equal/less/less-or-equal in comparison to the filter value. For null-valued singleton items always returns false.

Repository has limited support for multi-value properties on the left-hand side. See the section after this table.

Substring filter

property, limited multi-value support

property

single, non-null

-

Accepts if the filter value is a substring of one of the property values (optionally specifying if the property value should start or end with the filter value).

Repository has limited support for multi-value properties on the left-hand side. See the section after this table.

Ref filter

reference

-

single or multivalued (since 3.6), nullable

-

For null filter values: Accepts if the reference is empty.

For non-null filter values: Accepts if one of the reference values match the filter value (or one of filter values, if there are more than one), which means:

  1. OID matches,

  2. Referenced type matches, here null means "any type".

  3. Relation matches, null (or omitted) is equivalent to org:default. To match any relations, use PrismConstants.Q_ANY.

Org filter

(applicable to object as a whole)

-

single, non-null (or null with isRoot flag)

-

Accepts if the object is direct child or any descendant (this is configurable) of the referenced org. Alternatively, passes if the object is the root of the tree. As of 3.7.1 it can check the relation as well (see a note below).

Although technically not a Value filter, this filter can be seen as a special case of Ref filter using parentOrgRef as the item to be tested, and with some advanced options (scope, isRoot).

The Org filter relation is supported only for the directChildOf and childOf queries. It is silently ignored for parentOf queries. It is interpreted as a relation of the last (lowest) reference in the path, i.e. if we are looking for a user that is a child of org O1 with the relation of manager, we are looking for a user that is a manager of an org O2, which is either O1 itself or is any of its descendants.

See the dedicated section about Org filter at the end of this document and examples there.

InOid filter

(applicable to object/container value as a whole)

-

multivalued, non-null

-

Accepts if object OID (or ID for container values) is among filter values.

Question is if we should treat querying by ID/OID in the same way as querying by property, i.e. via Equal filter. ID/OID would be treated as a special kind of property. This would eliminate the need for InOid filter, but it might require deeper changes (e.g. there is no itemDefinition for ID/OID, etc). So, at least for midPoint 3.4, querying by ID/OID is done via InOid filter, not Equal filter.
Relation interpretation in Reference vs Org filter

Ref filter and Org filter can specify a relation to be looked for. It is specified as a relation on the reference value passed to the filter. However, for historical reasons, the null relation value is treated differently:

  • For Ref filter, null relation means default relation. If you need to check for any relation, you have to provide a value of q:any there.

  • For Org filter, null relation means any relation. Of course, q:any can be used as well, and is recommended for clarity.

Notes about value filters in repository queries

The following notes are based on the Native PostgreSQL repository implementation.

Repository engine is probably used most for the queries in midPoint, repository also provides the richest filtering support. But there are some inherent limitations:

  • Queries in midPoint can be totally arbitrary and some queries work faster and some may be slow. It is virtually impossible to optimize for all cases, given the filtering flexibility.

  • Queries are translated to the repository natural language - which is SQL. Things like collation can affect some operations, especially ordering and comparison of strings. Results can be different from expected, e.g. collation may be case-insesitive (default collation actually is).

  • Support for possible filter types (operations) for multi-value items depends on how they are stored in the DB. There is a full support for equals operation without any matching rule, regardless of the implementation. Support for substring and comparison operations is more tricky, depending on the storage mechanism for the item.

  • Multi-value items stored as text array columns (e.g. subtype) support all available operations. Collation can affect the expected results, as mentioned above.

  • Multi-value extension or attribute items stored in JSONB columns support most of the operations, depending on the type of the stored item. Text, numeric and date-time properties support all the operations. Enumerations do not support comparison operations, because the meaning is unclear, but EQ works as expected. Multi-value poly-strings currently (4.4+) support only EQ operation. Check also supported data types for extensions for more information.

Complex filters

Complex filters do contain other filters. They are:

Filter Description

And, Or, Not

Basic logical filters.

Type (type T, filter F)

Accepts iff the object is of type T and filter F passes.

Exists (item I, filter F)

Accepts iff there exists a value v of item I so that F(v) passes. This is useful e.g. to find an assignment with a given tenantRef and orgRef.

Logical filters

And, Or and Not filters are quite self-explanatory.

Type filter

An example: Imagine that the original query asked for an ObjectType. Then it is possible to set up Type filter with type=UserType, filter=(name equals "xyz") to find only users with the name of "xyz":

Example
<type>
  <type>UserType</type>
  <filter>
    <equal>
      <path>name</path>
      <value>xyz</value>
    </equal>
  </filter>
</type>

Exists filter

First of all, how should be individual value filters evaluated?

For example,

  • equal(name, 'xyz')

means "the value of object’s name is xyz". Simple enough.

In a similar way,

  • ref(assignment/tenantRef, oid1)

means "there is an assignment with a tenantRef pointing to oid1".

But what about this?

  • and(ref(assignment/tenantRef, oid1), ref(assignment/orgRef, oid2))

This one could be interpreted in two ways:

  1. There should be an assignment $a that has $a/tenantRef = oid1 and $a/orgRef = oid2.

  2. There should be assignments $a1, $a2 (potentially being the same) such that $a1/tenantRef = oid1 and $a2/orgRef = oid2.

Up to and including midPoint 3.3.1, the query is interpreted in the first way (one assignment satisfying both conditions).

But the interpretation should be following:

  • Each condition is interpreted separately.

  • So ref(assignment/tenantRef, oid1) should be read as "There is an assignment/tenantRef that points to oid1".

  • Therefore, the above complex filter should be interpreted in the second way: There should be assignments $a1, $a2 (potentially being the same) such that $a1/tenantRef = oid1 and $a2/orgRef = oid2.

If it’s necessary to say that one particular value of an item (presumably container) satisfies a complex filter, we use Exists filter.

The above complex filter - if needed to be interpreted in the first way - should be written like this:

  • exists ( assignment , and ( ref (tenantRef, oid1), ref (orgRef, oid2) ) )

Written in XML:

<exists>
  <path>assignment</path>
  <filter>
    <and>
      <ref>
        <path>tenantRef</path>
        <value>
          <oid> ...oid1... </oid>
        </value>
      </ref>
      <ref>
        <path>orgRef</path>
        <value>
          <oid> ...oid2... </oid>
        </value>
      </ref>
    </and>
  </filter>
</exists>

This feature is a part of midPoint 3.4 and above.

While EXISTS works as expected with complex combination of conditions for the common multi-value container (like assignment in the example above), NOT EXIST behavior works properly only with the new Native repository. Native repository (available since midPoint 4.4) translates EXISTS filter to SQL EXISTS and NOT is applied to the SQL EXISTS resulting in NOT EXISTS query, which works as expected.

EXISTS for the old Generic repo was implemented using LEFT JOIN which works fine for positive conditions, but does not work with NOT as usual in SQL. NOT EXISTS is interpreted as "any of the entries does not match the condition inside". See MID-7203.

Differences in filter interpretation

There are actually four "query engines" that interpret filters and queries:

Name Description Data types

repository

Interprets queries issued against repository objects.

almost all, except the ones described below

provisioning (connectors)

Interprets queries issued against resource objects, i.e. objects that reside on particular resources (AD, LDAP, CSV, …​).

ShadowType (some parts of them)

in-memory evaluator

Interprets queries/filters issued against objects already loaded into memory. Typically used for authorization evaluation.

all

These engines differ in capabilities and supported options. Due to historical reasons they might even interpret some filters in a slightly different way; this is unwanted and will be eventually fixed when discovered.

Let us summarize main differences here. Note that "ok" means "fully supported". "N/A" means "not applicable", i.e. not supported at all. The state is current as of midPoint 3.7.1.

Filter Repository Provisioning (connectors) In-memory

Equal

ok

Right-side items are not supported.

Right-side items are not supported.

Greater, Less

ok

N/A

N/A

Substring

ok

ok

ok

Ref

ok

N/A

ok

Org

ok

N/A

N/A

InOid

ok

N/A

ok

And, Or, Not

ok

ok

ok

Type

ok

N/A

supported but not much tested

Exists

ok

N/A

ok

Additionally, there are two parameters driving the behavior of Reference filters with null oid and targetType: oidNullAsAny and targetTypeNullAsAny. These are currently honored by memory and Native repository interpreters, not by Generic repository and connectors. These parameters are considered experimental and should be avoided as their meaning and/or existence is still debated.

General constraint for provisioning queries: It is not possible to mix both on-resource and repository items in one query, e.g. to query for both c:attributes/ri:something and c:intent.

For authoritative information about provisioning filter interpretation, see FilterInterpreter and related classes.

Filter examples

Filters can be created using Java API (traditional or fluent one) or via XML.

The following samples are taken from TestQueryConvertor class. XML versions are in the files named test*.xml in this directory.

Primitive filters

AllFilter

XML
<all/>
Traditional Java API
ObjectFilter filter = AllFilter.createAll();
Fluent Java API
ObjectFilter filter = prismContext.queryFor(UserType.class)
    .all()
    .buildFilter();

Note that QueryBuilder can return either whole query when .build() is used, or just a filter - with .buildFilter().

None and Undefined filters are created similarly.

Just for completeness, the whole query looks like this:

XML
<query xmlns="http://prism.evolveum.com/xml/ns/public/query-3">
    <filter>
        <all/>
    </filter>
</query>

The corresponding Fluent Java API call is:

Fluent Java API
ObjectQuery query = prismContext.queryFor(UserType.class)
    .all()
    .build();

To be concise, we’ll show only filters (no wrapping queries) in the following examples.

Value filters

EqualFilter

XML
<equal>
    <matching>polyStringOrig</matching>
    <path>c:name</path>
    <value>some-name</value>
</equal>
Traditional Java API
ObjectFilter filter = EqualFilter.createEqual(UserType.F_NAME, UserType.class, prismContext, PolyStringOrigMatchingRule.NAME, new PolyString("some-name", "somename"))

Fluent Java API:

Fluent Java API
ObjectFilter filter = prismContext.queryFor(UserType.class)
      .item(UserType.F_NAME).eqPoly("some-name", "somename").matchingOrig()
      .buildFilter();

Another example (we’ll show only XML and fluent Java API from this point on):

XML
<equal>
    <path>c:employeeType</path>
    <value>STD</value>
    <value>TEMP</value>
</equal>
Fluent Java API
ObjectFilter filter = prismContext.queryFor(UserType.class)
      .item(UserType.F_EMPLOYEE_TYPE).eq("STD", "TEMP")
      .buildFilter();

Comparing item to another item:

XML
<equal>
    <path>c:employeeNumber</path>
    <rightHandSidePath>c:costCenter</rightHandSidePath>
</equal>
Fluent Java API
ObjectFilter filter = prismContext.queryFor(UserType.class)
      .item(UserType.F_EMPLOYEE_NUMBER).eq().item(UserType.F_COST_CENTER)
      .buildFilter();

Comparisons

XML
<greater>
    <path>c:costCenter</path>
    <value>100000</value>
</greater>
Fluent Java API
ObjectFilter filter = prismContext.queryFor(UserType.class)
      .item(UserType.F_COST_CENTER).gt("100000")
      .buildFilter();

Or a more complex example:

XML
<or>
    <and>
        <greater>
            <path>c:costCenter</path>
            <value>100000</value>
        </greater>
        <less>
            <path>c:costCenter</path>
            <value>999999</value>
        </less>
    </and>
    <and>
        <greaterOrEqual>
            <path>c:costCenter</path>
            <value>X100</value>
        </greaterOrEqual>
        <lessOrEqual>
            <path>c:costCenter</path>
            <value>X999</value>
        </lessOrEqual>
    </and>
</or>
Fluent Java API
ObjectFilter filter = prismContext.queryFor(UserType.class)
      .item(UserType.F_COST_CENTER).gt("100000")
          .and().item(UserType.F_COST_CENTER).lt("999999")
      .or()
      .item(UserType.F_COST_CENTER).ge("X100")
          .and().item(UserType.F_COST_CENTER).le("X999")
      .buildFilter();

Substring filter

XML
<or>
    <substring>
        <path>c:employeeType</path>
        <value>A</value>
    </substring>
    <substring>
        <path>c:employeeType</path>
        <value>B</value>
        <anchorStart>true</anchorStart>
    </substring>
    <substring>
        <path>c:employeeType</path>
        <value>C</value>
        <anchorEnd>true</anchorEnd>
    </substring>
    <substring>
        <matching>polyStringOrig</matching>
        <path>c:name</path>
        <value>john</value>
        <anchorStart>true</anchorStart>
    </substring>
</or>
Fluent Java API
ObjectFilter filter = prismContext.queryFor(UserType.class)
      .item(UserType.F_EMPLOYEE_TYPE).contains("A")
      .or().item(UserType.F_EMPLOYEE_TYPE).startsWith("B")
      .or().item(UserType.F_EMPLOYEE_TYPE).endsWith("C")
      .or().item(UserType.F_NAME).startsWithPoly("john", "john").matchingOrig()
      .buildFilter();

Ref filter

"Canonical" form is the following:

XML
<or xmlns="http://prism.evolveum.com/xml/ns/public/query-3"
       xmlns:t="http://prism.evolveum.com/xml/ns/public/types-3"
       xmlns:c="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <ref>
        <path>c:resourceRef</path>
        <value xsi:type="t:ObjectReferenceType" oid="oid1" />
    </ref>
    <ref>
        <path>c:resourceRef</path>
        <value xsi:type="t:ObjectReferenceType" oid="oid2" type="c:ResourceType" />
    </ref>
    <ref>
        <path>c:resourceRef</path>
        <value xsi:type="t:ObjectReferenceType" oid="oid3" type="c:ResourceType" relation="test"/>
    </ref>
</or>

In Java:

PrismReferenceValue reference3 = new PrismReferenceValue("oid3", ResourceType.COMPLEX_TYPE);
reference3.setRelation(new QName("test"));

ObjectFilter filter = prismContext.queryFor(ShadowType.class)
      .item(ShadowType.F_RESOURCE_REF).ref("oid1")
      .or().item(ShadowType.F_RESOURCE_REF).ref("oid2", ResourceType.COMPLEX_TYPE)
      .or().item(ShadowType.F_RESOURCE_REF).ref(reference3)
      .buildFilter();

Semantics of individual 'or'-conditions is:

  1. resourceRef should contain: target OID = 'oid1', relation = (empty), and the type of target object (stored in the resourceRef!) can be any

  2. resourceRef should contain: target OID = 'oid1', relation = (empty), type of target (stored in the resourceRef!) must be 'ResourceType'

  3. resourceRef should contain: target OID = 'oid1', relation = 'test', and type of target (stored in the resourceRef!) must be 'ResourceType'

The reference target type, if used, must match exactly. So e.g. if the references uses RoleType, and the filter asks for AbstractRoleType, the value would not match.

It is suggested to avoid querying for target object type, if possible.

XML can be written also in alternative ways:

<or>
    <ref>
        <path>c:resourceRef</path>
        <!-- no xsi:type for 'value' element (this is not compliant with query-3 XSD) -->
        <value oid="oid1" />
    </ref>
    <ref>
        <path>c:resourceRef</path>
        <!-- no xsi:type, items stored as elements -->
        <value>
            <c:oid>oid4</c:oid>
            <c:type>c:ResourceType</c:type>
        </value>
    </ref>
</or>

Org filter

XML
<org>
    <isRoot>true</isRoot>
</org>
XML
<org>
    <orgRef>
        <oid>12345678-1234-1234-1234-0123456789abcd</oid>
    </orgRef>
    <scope>SUBTREE</scope> <!-- this is the default -->
</org>
XML
<org>
    <orgRef>
        <oid>12345678-1234-1234-1234-0123456789abcd</oid>
    </orgRef>
    <scope>ONE_LEVEL</scope>
</org>
Fluent Java API
ObjectFilter filter = prismContext.queryFor(OrgType.class).isRoot().buildFilter();
Fluent Java API
ObjectFilter filter = prismContext.queryFor(OrgType.class)
    .isChildOf("12345678-1234-1234-1234-0123456789abcd").buildFilter();
Fluent Java API
ObjectFilter filter = prismContext.queryFor(OrgType.class)
    .isDirectChildOf("12345678-1234-1234-1234-0123456789abcd").buildFilter();

InOid

XML
<inOid>
    <value>00000000-1111-2222-3333-444444444444</value>
    <value>00000000-1111-2222-3333-555555555555</value>
    <value>00000000-1111-2222-3333-666666666666</value>
</inOid>
Fluent Java API
ObjectFilter filter = prismContext.queryFor(UserType.class)
    .id("00000000-1111-2222-3333-444444444444",
        "00000000-1111-2222-3333-555555555555",
        "00000000-1111-2222-3333-666666666666")
    .buildFilter();

This one selects container values with ID 1, 2 or 3, having owner (object) with OID of "00000000-1111-2222-3333-777777777777".

XML
<and>
    <inOid>
        <value>1</value>
        <value>2</value>
        <value>3</value>
    </inOid>
    <inOid>
        <value>00000000-1111-2222-3333-777777777777</value>
        <considerOwner>true</considerOwner>
    </inOid>
</and>
Fluent Java API
ObjectFilter filter = prismContext.queryFor(UserType.class)
      .id(1, 2, 3)
      .and().ownerId("00000000-1111-2222-3333-777777777777")
      .buildFilter();

Logical filters

An artificial example:

XML
<and>
    <or>
        <all/>
        <none/>
        <undefined/>
    </or>
    <none/>
    <not>
        <and>
            <all/>
            <undefined/>
        </and>
    </not>
</and>
Fluent Java API
ObjectFilter filter = prismContext.queryFor(UserType.class)
      .block()
         .all()
         .or().none()
         .or().undefined()
      .endBlock()
      .and().none()
      .and()
         .not()
            .block()
               .all()
               .and().undefined()
            .endBlock()
      .buildFilter();

Type filter

XML
<type>
    <type>c:UserType</type>
    <filter>
        <equal>
            <path>c:name</path>
            <value>somename</value>
        </equal>
    </filter>
</type>
Fluent Java API
ObjectFilter filter = prismContext.queryFor(ObjectType.class)
    .type(UserType.class)
        .item(UserType.F_NAME).eqPoly("somename", "somename")
    .buildFilter();

Exists filter

An example: Find all certification cases that have at least one missing response for a given reviewer.

So we are looking for a certification case, that has a decision D for which:

  1. D’s reviewer is the given one,

  2. D’s stage number is the same as case’s stage number (because certification case contains decisions from all the stages),

  3. D’s response is either null or 'noResponse'

It looks like this in XML:

XML
<exists>
    <path>c:decision</path>
    <filter>
        <and>
            <ref>
                <path>c:reviewerRef</path>
                <value oid="123456" xsi:type="t:ObjectReferenceType"/>
            </ref>
            <equal>
                <path>c:stageNumber</path>
                <rightHandSidePath>../c:currentStageNumber</rightHandSidePath>
            </equal>
            <or>
                <equal>
                    <path>c:response</path>
                </equal>
                <equal>
                    <path>c:response</path>
                    <value>noResponse</value>
                </equal>
            </or>
        </and>
    </filter>
</exists>

And in Java:

ObjectFilter filter = prismContext.queryFor(AccessCertificationCaseType.class)
        .exists(AccessCertificationCaseType.F_DECISION)
        .block()
            .item(AccessCertificationDecisionType.F_REVIEWER_REF).ref("123456")
            .and().item(AccessCertificationDecisionType.F_STAGE_NUMBER)
                .eq().item(PrismConstants.T_PARENT, AccessCertificationCaseType.F_CURRENT_STAGE_NUMBER)
            .and().block()
                .item(AccessCertificationDecisionType.F_RESPONSE).isNull()
                .or().item(AccessCertificationDecisionType.F_RESPONSE).eq(NO_RESPONSE)
            .endBlock()
        .endBlock()
        .buildFilter();

Expression filter

XML
<substring>
    <matching>polyStringNorm</matching>
    <path>name</path>
    <expression>
        <script>
            <code>
                return 'C';
            </code>
        </script>
    </expression>
    <anchorStart>true</anchorStart>
</substring>

This example returns all objects with a name starting with "C".

Date filtering

XML
<and>
    <greater>
        <path>extension/EndDate</path>
        <expression>
            <script>
                <code>
                    return basic.parseDateTime('yyyy-MM-dd', (basic.currentDateTime().getYear()-1) + '-12-31');
                </code>
            </script>
        </expression>
    </greater>
    <less>
        <path>extension/EndDate</path>
        <expression>
            <script>
                <code>
                    return basic.parseDateTime('yyyy-MM-dd', basic.currentDateTime().getYear() + '-01-02');
                </code>
            </script>
        </expression>
    </less>
</and>

This example returns all objects with extension attribute "EndDate" (type of XMLGregorianCalendar), which is set since 31 Decenber last year to 01 January of this year.

Special symbols in item paths (.., @, #)

This section is not up-to-date with midPoint 4.4 LTS version. The use of @ to traverse to reference target ("dereference") is safe and well-supported. Using .. and # is experimental, was introduced for internal reasons and the support is limited.

An example: Find all active certification cases for a given reviewer.

An active certification case is one that is part of a campaign that is in a review stage, and whose current stage number is the same as the owning campaign current stage number.

XML
<and>
    <ref>
        <path>c:currentReviewerRef</path>
        <value oid="1234567890" type="c:UserType" xsi:type="t:ObjectReferenceType"/>
    </ref>
    <equal>
        <path>c:currentStageNumber</path>
        <rightHandSidePath>../c:stageNumber</rightHandSidePath>
    </equal>
    <equal>
        <path>../c:state</path>
        <value>inReviewStage</value>
    </equal>
</and>

The .. symbol denotes "owning campaign".

ObjectFilter filter = prismContext.queryFor(AccessCertificationCaseType.class)
        .item(F_CURRENT_REVIEWER_REF).ref(reviewerRef)
        .and().item(F_CURRENT_STAGE_NUMBER).eq().item(T_PARENT, AccessCertificationCampaignType.F_STAGE_NUMBER)
        .and().item(T_PARENT, F_STATE).eq(IN_REVIEW_STAGE)
        .buildFilter();

PrismConstants.T_PARENT is the QName for .. path segment.

Following example uses @ symbol to dereference linkRef to ShadowType in user object. This allows e.g. filtering users that have projection on specified resource. Please note, that @ has limitation towards general (any object type) usage and will work with statically defined types like ObjectType, FocusType, ShadowType.

XML
<filter>
    <ref>
        <path>linkRef/@/resourceRef</path>
        <value oid="7754e27c-a7cb-4c23-850d-a9a15f71199a"/>
    </ref>
</filter>

Another examples:

XML
<filter>
    <equal>
        <path>assignment/targetRef/@/name</path>
        <value>CN=AD-group,OU=Groups,DC=evolveum,DC=com</value>
    </equal>
</filter>

UserType: linkRef/@/resourceRef/@/name contains 'CSV' (norm).

Java
ObjectQuery query = prismContext.queryFor(UserType.class)
    .item(UserType.F_LINK_REF, PrismConstants.T_OBJECT_REFERENCE,
        ShadowType.F_RESOURCE_REF, PrismConstants.T_OBJECT_REFERENCE, F_NAME)
    .containsPoly("CSV").matchingNorm().build();

Paging

Limiting the number of returned entries, offset, etc., can be configured using paging. Following table shows paging options:

Option Possible values Default value Description

orderBy

property path (e.g. name)

arbitrary search

Property by which the results should be sorted. Only one property is supported by the query language at this moment. The item to order by should be a single-value item, logically, ordering by multi-value item is ambiguous.

ascending

ascending/descending

ascending

Direction of ordering (ascending or descending). Only valid if orderBy is specified.

offset

any Integer

0

The index of the first returned entry, starting with zero. If 0 specified, resulting list will start with first entry. If 1 specified, resulting list will start with second entry.

maxSize

any Integer

2147483647

The maximum number of entries returned. The operation may return specified number of entries or less.

Following is the example for using paging in the query.

Paging example
<q:query xmlns:q="http://prism.evolveum.com/xml/ns/public/query-3">
    <q:filter>
        <q:equal>
            <q:path>activation/administrativeStatus</q:path>
            <q:value>enabled</q:value>
        </q:equal>
    </q:filter>
    <q:paging>
        <q:orderBy>name</q:orderBy>
        <offset>0</offset>
        <maxSize>10</maxSize>
    </q:paging>
</q:query>

Using example above will return first 10 records ordered by name where administrativeStatus is set to enabled.

Ordering objects by items inside multi-value container have unclear semantics and is not supported.

Org filter

Java Query API is used in this section for brevity.

First we reiterate the information from above:

  • Org filter is used for the whole object. Query can return organizations or other types assignable to organizations, depending on the filter specifics (see the table below).

  • Org filter works only for repository queries.

  • With is(Direct)ChildOf filters it is possible to filter on relation value as well. If relation is not stated, it matches any relation (this is different from normal ref filters).

  • Parameter of the is(Direct)ChildOf and isParentOf is an OID of another organization. With isParentOf it’s not possible to search for organizations above, let’s say, a user.

Org filter Possible queryFor type Parameter Notes

isRoot

OrgType only, not defined on non-org query

none

Matches orgs without any parent organization. Does not take any parameter.

isChildOf

OrgType or any type assignable to organization

OrgType specified by OID or as PrismReferenceValue

Matches any object that is directly or indirectly under the organization specified in the parameter. If relation is specified (isChildOf with PrismReferenceValue parameter must be used for this) it filters the objects with their parent-org reference having the specified relation (the reference closest to the returned object, see picture below for example).

Query does not return object used as a parameter (object is not considered a child of itself).

isDirectChildOf

OrgType or any type assignable to organization

OrgType specified by OID or as PrismReferenceValue

Matches any object that is directly under the organization. Technically, this means that the returned object must have a parent-org reference with the target pointing to the organization specified in the parameter of the filter. Just as in isChildOf, optional relation can be specified, here the semantics is obvious as there is only a single ref leading from possibly returned object to the org specified in the parameter.

Query does not return object used as a parameter (an org is not considered a child of itself).

isParentOf

OrgType only, not defined on non-org query

OrgType specified by OID or as PrismReferenceValue

Matches any organization that is direct or indirect parent (ancestor) of the organization specified in the parameter. It is not possible to filter by relation, it is ignored if specified.

Query does not return object used as a parameter (an org is not considered a parent of itself).

Few examples of matching and not-matching filters are shown in the following picture:

org filters

Relation matching examples

Let’s consider the following filter now:

ObjectFilter filter = prismContext.queryFor(ObjectType.class)
    .isChildOf(prismContext.itemFactory().createReferenceValue(oidOrg1, relationX))
    .buildFilter();

Let’s use this simple organization structure where red arrows designate parent-org references with X relation:

org filters relation

Query with this filter returns objects with red border because the parent-org references they own have relation X (these would appear in object’s serialized form as parentOrgRef elements). Other objects have references with different relations and are not returned. If isChildOf(oidOrg1) was used instead without specifying the relation, query would return all objects under ORG 1.

Now let’s change the object type for the query to UserType:

ObjectFilter filter = prismContext.queryFor(UserType.class)
    .isChildOf(prismContext.itemFactory().createReferenceValue(oidOrg1, relationX))
    .buildFilter();

The query returns User 1-1-1_E and User 1-1_B because only these have the right relation in their immediate (owned) parent-org reference and are of the requested type.

Similarly, only the orgs with red border would be returned if OrgType was used instead.

Only the parent-org reference owned by the potentially matching object is consulted. This does not mean that only leaves of the tree are returned, as demonstrated by ORG 1-1 being returned (because its parent-org ref has the specifiec X relation).

Notice, that User 1-1-1_D also has parent-org ref with relation X somewhere on the path to the ORG 1 (parameter of the isChildOf filter), but this does not matter as the reference it ownes (the one pointing to Org 1-1-1) has different relation.