Reference searching

Last modified 01 Mar 2023 23:52 +01:00
Table of Contents
This is a design document. For reference on this topic, check the section in midPoint Query.

It is already possible to search objects and containers. It is also possible to search by reference value and its target. But to be able to search accesses (represented by roleMembershipRef) we need to search references as well. This means, that the number of results matches the number of found references. This is similar to container search, e.g. we can search multiple assignments inside a single object (or accross multiple objects).

The missing parts are:

  • How to provide the search instructions?

  • How to filter based on the owner of the reference? (We already can filter by the reference target, this is a solved problem.)

  • How to construct the result set?

Owning object…​ or container?

For roleMembershipRef the situation is very simple. Object owns the reference and its item path is roleMembershipRef. But in the future we may want to search for case/workItem/assigneeRef in access certification campaign. What is the owner of this reference? Is it the access certification campaign and the ref path is case/workItem/assigneeRef? Or is it the AccessCertificationWorkItemType with the ref path assigneeRef? The owning object is clearly an object, not a container. But when we talk about parent it is the container - just like it works for containers now.

Currently, refs do not have parent searchable because it wasn’t needed when the search does not start on the reference.

Search instructions

We need to be able to specify:

  • Which reference (table) to search - what goes to the select clause (search root).

  • Additional filter for the target of the reference - which is already a solved problem.

  • Additional filter for the owner and/or parent of the reference - probably parent is more consistent with existing capabilities.

  • Pagination details.

Specifying search root

To specify the reference there is a couple of options:

  • We can introduce an enumeration of supported reference searches and use these values directly, e.g. ACCESS_CERT_WI_ASSIGNEE_REF would search for case/workItem/assigneeRef on access certification campaings.

  • We can specify the starting object and the item path to the reference, e.g. AccessCertificationCampaignType and case/workItem/assigneeRef.

  • We can specify the parent object/container, and then single-segment item path, e.g. AccessCertificationWorkItemType and assigneeRef. This is also ambiguous for assignments, but can be clarified by ownedBy.

When it comes to providing the item path to the ref, it can be explicit, or part of the ref filter (see the next section).

(RR) I believe that any of the options is OK as long as one is chosen and not much flexibility is allowed to keep it simple. In theory, one would argue, that both options with type and path are OK, and even AccessCertificationCaseType with workItem/assigneeRef is an equivalent option. But I’d allow only object+path or parent container+path from these, not any combination.

Specifying the ref filter

Using RefFilter as a search filter seems a natural choice. This can provide also the itemPath to the reference (relative to the allowed owner or parent type). Ref filter can also contain the target filter (for the reference target), as already supported.

Parent/owner filter

For refs that are defined directly in the object type, that is single-segment path from the object type, the parent and owner are the same and there is no complication. The filtration can be achieved by an optional separate filter, which - if provided - is applied to the joined owner.

For any deeper ref (e.g. metadata/modifierRef or workItem/originalAssigneeRef) it is much flexible to go up the hierarchy step by step and use some kind of parent filter - similar to OwnedByFilter. The reason is, that this way we can clearly specify filter conditions not only on the object, but also on the parent container.

Starting this on the owner object (e.g. a CaseType for workItem/originalAssigneeRef) makes it already disconnected. How would we express our interest only in those refs owned by a CaseWorkItemType with, for instance, this and that stage number? Going from the object down could be interpreted as "we want workItem/originalAssigneeRef from a case that has (any) work item with such a stage number - which may not be the work item with the searched reference. Clearly saying that the parent of the ref should have such a stage number leads to no such disconnection.

Reference Query?

This is early thought experiment. See the further sections for final solution.

With the need to provide all the things above plus pagination it is probably best to pack it in a dedicated type. The structure could be:

ReferenceQuery:
- filter (RefFilter, mandatory unless itemPath is provided explicitly)
- parentFilter (equivalent of OwnedByFilter.filter)
- paging
- allowPartialResults

OwnedByFilter provides also type and path:

  • path is needed for containers mainly to distinguish between assignemnts and inducements, but this is not a problem for reference search - and the path is actually part of the main (ref) filter;

  • type could narrow down the initial type provided to the searchRefs call - but one could just as well provide the narrower type directly (e.g. UserType instead of AssignmentHolderType).

This makes only the filter part of the OwnedByFilter relevant for this case. Considering this, it is questionable whether we want to embed this filter into RefFilter directly. On the one hand it would make ref filter more consistent with the container search. On the other hand it is not useful on the RefFilter for any other type of search, because normally we get to the refs through their owning object/container(s).

Final solution is mandatory ownedBy next to ref. Owned by directly inside ref is not good for a couple of reasons:

  • If we want all refs of some type we would do that with no (null) ref filter. Where would the additional ownedBy go then?

  • As mentioned, ownedBy is otherwise not useful for the ref filter.

  • Owned-by filter is common for the search, so it can’t be in (some) ref filter if we allow combination of ref filters (AND/OR).

The parentFilter/ownedBy filter is also useful while applying security - at least when the parent is an object for which we already support the security pre-processing.

Eventual ReferenceQuery is quite similar to existing ObjectQuery with the following differences:

  • Initially, it is nice to type the filter as RefFilter - but this does not allow AND/OR.

  • Additional parentFilter is needed or ownedBy support inside the RefFilter.

Two vs one filter

But there are also a few good reasons to have ownedBy part inside the RefFilter:

  • We don’t need to construct two separate filters for one reference search.

  • ObjectQuery could be used as is, or perhaps with minimal type parametrization to enforce RefFilter.

So with ownedBy in RefFilter there is only a difference in the type of the root filter. But even ObjectFilter for reference search can make sense, because that way we can OR or AND multiple ref filters, not to mention top level NOT.

When complex filters are allowed, though, it doesn’t make sense to use itemPath to the ref from RefFilter. With multiple ref filters inside a complex filter we still need just a single itemPath to the ref and a very likely just one parentFilter, not ownedBy for each RefFilter. The itemPath problem can be solved by and explicit itemPath - and then ignoring the itemPath in the ref filters applicable to the root of the search. WARNING: itemPath can contain @/otherRef and not be relevant to the search root!

Can we limit ourselves now to a single RefFilter on the top level? This still allows using complex target/owner filters, but not disjunct combination of ref types/relations.

Usage examples

Just like the type for other queries is provided separately (and clearly identifies the table for select), the identification of the ref table should also be explicit, hence probably out of the top level ref filter. Then we can ignore the itemPath inside ref filters and this also solves one of the questions if complex filters are allowed. This explict itemPath can be part of the ReferenceQuery type or explicit parameter, or implied by the new enum type of supported ref searches (mentioned previously in Specifying search root).

Knowing what kind of reference search is required is also important for definitions during filter construction.

Java fluent API

Alternative 1 - existing API without any changes, but with two filters constructed:

var userTypeQuery = prismContext.queryFor(UserType.class);
ObjectQuery refQuery = userTypeQuery
        .ref(UserType.F_ROLE_MEMBERSHIP_REF)
        .item(F_NAME).eq("actual-role-name")
        .maxSize(5)
        .build();
ObjectFilter parentFilter = userTypeQuery.id("user-oid-here").buildFilter();
SearchResultList<ObjectReferenceType> objectReferenceTypes =
        repositoryService.searchReference(UserType.class, refQuery, parentFilter);

WIP: Alternative 2 - single filter with ownedBy support in RefFilter. How to distinguish between (or start/stop) target filter and ownedBy filter? We probably don’t want some magic like "AND with ref and owned by filter is interpreted in this way"…​ or do we? EDIT: Actually, this is on the way to the final solution with AND containing mandatory ownedBy filter.

ObjectQuery refQuery = prismContext.queryFor(UserType.class)
        .ref(UserType.F_ROLE_MEMBERSHIP_REF) // returns some new S_RefEntry state
            .targetFilter()
            //.block() // optional, needed for OR/AND
                .item(F_NAME).eq("actual-role-name")
            //.endBlock() // optional, needed for OR/AND
            //.and() //possible but useless in constrained S_RefEntry context
            .ownedBy(UserType.class)
            //.block() // optional, needed for OR/AND
                    .id("user-oid-here")
            //.endBlock() // optional, needed for OR/AND
        .endRef() // important, pair to ref(...)
        .build();
SearchResultList<ObjectReferenceType> objectReferenceTypes =
        repositoryService.searchReference(UserType.class, refQuery);
ObjectQuery refQuery = prismContext.queryForReference(UserType.class, F_ROLE_MEMBERSHIP_REF)
    // already contains owned by filter - but how to customize it?

ObjectQuery refQuery = prismContext.queryForReferenceOwnedBy(UserType.class, F_ROLE_MEMBERSHIP_REF)
    block().endBlock() // empty owner subfilter... can we
    .and()
    ...

ObjectQuery refQuery = prismContext.queryForReference()
    .ownedBy(UserType.class, F_ROLE_MEMBERSHIP_REF) // here the defs are populated
    .block().endBlock()
    .and()
    .ref(., ...)

Axiom

Axiom only creates the filter, not the query, but that’s OK.

Let’s just expore the ref filter allowing owned by directly:

. matches (
    // @ represents ref target, target filter is inside (...)
    @ matches (
        name = "actual-role-name" ) )
and
. ownedBy (
    @type = UserType
    and . inOid 'user-oid-here' )

Constructing result set

  1. Select refs, this can be low-level repo based List<MReference>.

  2. Collect unique owner OIDs.

  3. Select owner objects.

  4. Crawl the objects to extract the result references from them.

Considerations:

  • When do we want/need refs from objects? For one user we don’t need it.

  • Value metadata are not in returned MReference - if they need to be part of the returned data, we need to get the ref from the loaded object. This, again, may be unnecessary for cases when GUI works with a single object.

These are all low-priority problems, at this moment we want either refs only, or refs from objects, without any Get options customization for the object. What should be the default behavior?

Can we extract metadata in expressions? Currently, probably not…​ just adding support for @metadata segment may not be enough because of multi-value nature of metadata.

Other problems

  • How to apply security on the model layer? This probably requires the owner filter part. Do we need to solve it in the first implementation? Currently, applySchemasAndSecurityToContainers is called only for AccessCertificationCampaignType, so other container types don’t have this covered either. While preProcessOptionsSecurity is called for containers, preProcessQuerySecurity is not called for the container query at all, only for the object search.

MEETING

Example: "Show me all membership refs on a user." This only requires ownedBy filter, but ref filter must work like "all". This is probably null ref filter - which cannot contain ownedBy! - it’s not ref filter with no value.

It seems we want ownedBy independent of the ref (or complex AND/OR) filter.

Another idea:

If ownedBy is mandatory on the top level, it can specify the path to the ref better than any of the ref filters (if any is even there). This would allow searchRefs(ObjectQuery, options…​) without any type parameter.

Owned by and OR of two relations:

// ownedBy added automagically by GUI where expected
. ownedBy ( @type = UserType and @path = roleMembershipRef)
and (
  . matches ( relation = manager )
  or . matches (relation = default)
)
and @ matches ( archetypeRef/@/name = "BusinessRole")
. ownedBy ( @type = UserType and @path = roleMembershipRef)
and (
  . matches ( relation = manager )
  or . matches (relation = default)
)
and @ matches ( archetypeRef/@/name = "BusinessRole")

Meeting Conclusions

  • Model/repo API something like:
    SearchResultList<ObjectReferenceType> searchReferences(ObjectQuery, options…​)

  • Query must be AND and contain top level ownedBy filter (check provisioning query interpretation) and additional ref filter(s).

  • POC fluent for fluent API (e.g. queryForReferenceOwnedBy(…​)).

  • Axiom will need to support non-container definitions too. (Tony)

Was this page helpful?
YES NO
Thanks for your feedback