Manage Dead Shadows

Last modified 31 Oct 2025 14:44 +01:00
Table of Contents

This page overviews dead shadows in midPoint and how to manage them.

Dead shadows in midPoint are remnants of deleted accounts. They help prevent accidental re-creation of those accounts, and ensure data consistency.

Refer to Dead shadow for more details on what dead shadows are and why they exist.

Manage dead shadows

Dead shadows are quite useful, for example, for auditing and historical traceability, and they usually do not pose any performance problems.

Dead shadows are automatically cleaned up (deleted) after 7 days.

You can configure the retention period for dead shadows by setting the deadShadowRetentionPeriod property in your resource. To disable the dead shadows functionality altogether, set the deadShadowRetentionPeriod property to zero:

<resource>
    ...
    <consistency>
        <deadShadowRetentionPeriod>PT0S</deadShadowRetentionPeriod>
    </consistency>
    ...
</resource>

Implementation notes

  • A resource object can only have one "live" shadow but multiple dead shadows.

  • Dead shadows are never reused.

    • Once a shadow is dead, it stays dead. There is no way to resurrect it.

    • There can only be one ADD operation for a shadow and one DELETE operation.

    • If a deleted account gets re-created, a new shadow is created. There is a reason for that: a new account is a completely new account. It may have a different (primary) identifier, e.g., entryUUID generated by the resource. Therefore, creating a new shadow is more than justified.

TODO: refresh operation, get operation, forceRefresh option, reshreshOnRead

Shadow lifecycle and flags

There are two flags that define shadow lifecycle: dead and exists. They specify midPoint’s best knowledge about the state of a resource (not its "snapshot" used in semi-manual resources).

State shadow dead flag shadow exists flag shadow pendingOperation Resource connector operation (e.g., ticket, async operation) Resource database Resource snapshot (CSV export) Description

Proposed

false

false

Requested operation (ADD)
Status: REQUESTED

No operation

Not present

Not present

Operation is requested but it was not started yet. We are processing the request.
This is used mostly to detect uniqueness conflicts (to "reserve" identifiers).

Conception

false

false

Requested operation (ADD)
Status: EXECUTION_PENDING or EXECUTING

ADD operation "open" (executing)

Not present
(but being created)

Not present

A signal to create an account has been sent and it is being executed.

Gestation

false

true

Requested operation (ADD)
Status: COMPLETED/SUCCESS
Operation in its grace period

ADD operation "closed" (completed)

Present
(most likely)

Not present

This is a "quantum" state: shadow is alive but not yet alive at the same time. It probably already exists in the resource (hence exists=true). But it does not exist in the snapshot yet.
Gestating shadows will not appear in resource searches. This should not be a problem for reconciliation because shadows should be linked and they will be processed by reconciliation anyway.
In case that the ADD operation was a failure, the shadow should instantly become a tombstone.

Life

false

true

No ADD operation (or operation over grace period)
There may be MODIFY operations.

No operation or modify operations only

Present

Present

This is the normal state. The shadow exists, everything works as expected. There are no quantum effects or controversies.
In case that the object is not present in the snapshot, the shadow becomes a tombstone.

Reaping

false

true

Requested operation (DELETE)
Status: EXECUTION_PENDING or EXECUTING

DELETE operation "open" (executing)

Present
(but being deleted)

Present

Signal to delete account was sent. It is being executed.

Corpse

true

false

Requested operation (DELETE)
Status: COMPLETED/SUCCESS
Operation in its grace period

DELETE operation "closed" (completed)

Not present
(most likely)

Present

A.k.a. Schroedinger’s shadow.
This is a "quantum" state: shadow is dead but also alive at the same time. It is probably already deleted in the resource (hence exists=false). But it still exists in the snapshot.
Corpse shadows will appear in resource searches - even though the shadow is marked as dead=true.
TODO: what to do if DELETE operation was a failure? Return to life? Or do we need a "zombie" state?

Tombstone

true

false

No operations, or only operations over grace period.

No operation

Not present

Not present

Shadow is dead. Nothing remains. No resource object, no object in the snapshot. Just this stone on a grave remains. And it will also expire eventually.
Tombstone shadows will not appear in resource searches - because they do not exist on the resource. But they will work with get operations. And they can be searched with noFetch.
This is the terminal state. Shadow stays dead and it cannot be resurrected.

Shadow graveyard

Getting an object will always return a shadow if there is one. Even if it is a tombstone. An ObjectNotFound exception is thrown only if there is nothing to return: no resource object and no shadow. Therefore, clients cannot assume that a resource object exists if an getObject() operation returns something. Clients should always check shadow lifecycle flags (dead, exists).

TODO: cleanup of dead shadows. grace period, operation retention period, dead shadow retention period

Semi-manual "quantum" cases (Schroedinger’s shadow)

Semi-manual connectors (CSV) introduce some additional complications due to the nature of their operation. Because those connectors work with exported snapshots of the resource data, there is always some delay between the actual state of the resource and the state known to midPoint. This leads to some "quantum" states where midPoint is not sure whether the account exists or not.

There are two such cases:

  • An administrator creates the account directly in the target system and closes the ticket. However, the exported CSV snapshot has not yet been refreshed, so the new account is not present in the file. From midPoint’s perspective, the create operation succeeded, so the shadow should be alive. But reading from the resource (the outdated CSV) will fail because the account is not there — meaning the shadow should not be alive.

  • An administrator deletes the account in the target system and closes the ticket. However, the exported CSV snapshot has not yet been updated, so the deleted account still appears in the file. From midPoint’s perspective, the delete operation succeeded, so the shadow is marked as dead. But since the account is still visible in the snapshot, a resource search will still return it, meaning the shadow should not be dead. This creates a Schrödinger’s shadow situation: during the grace period, midPoint can still fetch data from the resource; after the grace period expires, the shadow becomes fully dead.

When searching through the resource, we are in fact searching through the CSV and the account-that-should-be-dead-but-it-is-not-dead-yet will be part of search results. In that case:

  • If there is a pending delete operation in the dead shadow, we return the dead shadow - even if the account is still "alive" in the snapshot (CSV).

  • If there is no pending operation (or operation over the grace period), provisioning will stop playing Schroedinger here. The dead shadow will remain dead and provisioning will create a new live shadow for the account. A discovery will run and all that usual stuff.

Was this page helpful?
YES NO
Thanks for your feedback