Shadow Integrity Check

Last modified 22 Apr 2021 17:31 +02:00

In order to diagnose and fix specific problems with shadow objects in repository, a new task type was created: Shadow Integrity Check Task.

How to use it: just import the following task (taken from samples/tasks/task-check-shadow-integrity.xml).

<task xmlns="">
    <name>Shadow Integrity Check</name>
    <extension xmlns:mext="">
            <q:paging xmlns:q="">
                <q:orderBy>name</q:orderBy>     <!-- in order to work with repositories that implement iteration by paging (so the order is not changed via fix operations) -->
    <ownerRef oid="00000000-0000-0000-0000-000000000002" type="UserType"/>

There are the following configuration options, in task extension container:

  1. objectQuery: Specifies which shadows to process. The default is all shadows. Note that for databases that implement iterative searches via paging (taking e.g. records 1-100, then 101-200, etc) it is advisable to define an order-by criterion, which is independent on fixes carried out. E.g. if the fix applied is the normalization of attributes, then the shadow object name is OK, because it is not changed in the process. Such databases are H2 and MySQL.

  2. diagnose and fix: Specify which problems to diagnose (default = everything except "fetch") and which to fix (default = none).

Currently supported checks are:

Keyword Description Support


Checks whether all identifiers in a shadow are in a normalized form with regards to matching rule specified in the schemaHandling section. (Note: before running this, please check that you have consistently specified matching rule for all intents of a given kind of objects. In future, we’ll check that setting automatically, but currently we do not.)

diagnose + fix


Checks whether there are more shadows (of given resource+kind) bound to specific identifier value, e.g. icfs:name or icfs:uid or whatever.

diagnose + fix (\*)


Checks whether shadow has intent specified. Issues a warning if it has not. When fixing, fetches the resource object and tries to determine correct intent.

diagnose + fix


Checks for extra data in shadow activation container. As of midPoint 3.3, we store only enableTimestamp, disableTimestamp, archiveTimestamp and disableReason there. All other data (e.g. administrativeStatus) have to be fetched from the resource on the fly. This check ensures that the superfluous data are not stored in the shadows, and removes those data that are there (e.g. after migration from pre-3.3 versions of midPoint).

diagnose + fix


Checks whether shadow has at least one owner + if the synchronization situation (LINKED) is consistent with owners found - LINKED shadow has to have at least one owner, not-LINKED shadow has to have 0 owners.

diagnose only


Checks whether the corresponding object can be fetched from the resource. Because of the performance implications, this check is not carried out by default.

diagnose only


Checks whether resource pointed to by the shadow really exists, including whether resource reference is set at all. (Actually, these checks cannot be turned off.) Fix is done by removing the shadow, so please use with care.

diagnose + fix

(\*) Note that the test checking for uniqueness is implemented purely in RAM, so it works for shadow sets of reasonable size (e.g. 100K entries, depending on amount of your heap). It is faster and simpler to implement. If necessary, just log a JIRA issue to implement more general solution that would work with shadow sets of any size.

The task produces a report that is written to midPoint log file. It looks like this:

2015-09-03 12:39:23,464 [] [midPointScheduler_Worker-6] INFO (com.evolveum.midpoint.model.impl.integrity.ShadowIntegrityCheckResultHandler): Shadow integrity check finished. It was run with the configuration:
- normalization  diagnose=true,    fix=true
- uniqueness     diagnose=true,    fix=true
- intents        diagnose=true,    fix=true
- owners         diagnose=false
- fetch          diagnose=false

dryRun = true

2015-09-03 12:39:23,464 [] [midPointScheduler_Worker-6] INFO (com.evolveum.midpoint.model.impl.integrity.ShadowIntegrityCheckResultHandler): Results:
    Shadows processed: 3020 (1 resources),
    Shadows with no problems: 2875
    Shadows with warnings: 141
    Shadows with errors: 4
     - Non-normalized identifier value: 0 cases; fixed (if not run in dry-run mode) 0 cases.
     - No resource ref or OID: 0 cases.
     - Cannot get resource object: 0 cases.
     - No kind specified: 0 cases.
     - No intent specified: 145 cases (145 shadows).
     - No resource refined schema: 0 cases.
     - Cannot get resource refined schema: 0 cases.
     - No object class refined schema: 0 cases.
     - Cannot fetch resource object: 3 cases (3 shadows).
     - Multiple owners: 0 cases.
     - Linked shadow with no owner: 0 cases.
     - Not linked shadow with an owner: 0 cases.
     - Other failure: 0 cases.
     - Cannot apply fix: 4 cases (4 shadows).

2015-09-03 12:39:23,464 [] [midPointScheduler_Worker-6] INFO (com.evolveum.midpoint.model.impl.integrity.ShadowIntegrityCheckResultHandler): Uniqueness report:
Duplicate shadows detected: 1, deleted: 0
Duplicates for resource: .............. (OID:..............), kind = ENTITLEMENT, identifier = {}name:
 - value: cn=.............., shadows: 2
   - shadow: CN=.............. (OID:16fc34ab-5acc-4320-8db1-6f707d2ee9ac); sync situation = null
     - name = [cn=..............]
     - uid = [<GUID=5a586bc978e98245ba529adbe83ced57>]
   - shadow: CN=.............. (OID:9b76adf6-0878-4fd3-a5ec-d9d33505ffa8); sync situation = LINKED
     - name = [cn=..............]
     - uid = [<GUID=91ebc50e25056a49a562ea7492467433>]
   --> deleted redundant shadow (skipped because of dry run) shadow: CN=.............. (OID:16fc34ab-5acc-4320-8db1-6f707d2ee9ac)

2015-09-03 12:39:23,464 [] [midPointScheduler_Worker-6] INFO (com.evolveum.midpoint.model.impl.util.AbstractSearchIterativeTaskHandler): Finished Shadow integrity check (Task(id:1441204963732-0-1, name:A Shadow Integrity Check, oid:db2ce226-c13f-4731-ab2e-9bad11223fe0)). Processed 3020 objects in 180 seconds, got 3 errors. Average time for one object: 56.21755 milliseconds (wall clock time average: 59.927483 ms).