Shadow Refresh Task

Last modified 04 Sep 2024 12:44 +02:00
Table of Contents

Introduction

There are situations, when shadow objects do not entirely reflect the reality. Such situations are to be expected, as those "shadows" are actually copies of the data that originate from other systems. The shadow objects also contain pending operation data that naturally change in time. And there are also cases when the data model evolves and data in the "shadows" need to be updated.

All such situations mean that shadow objects need to refreshed. Refresh means that midPoint makes sure that a shadow is internally consistent and to execute all operations that need to be executed in order to re-gain consistency. Shadow refresh will try to finish pending operations, e.g. re-try failed operations. Refresh will update the status of manual operations. And so on. Refresh task will also make sure that application-level indexes for shadow objects are right. Therefore it can be useful in migration scenarios or in cases that system configuration was changed and stored data need to be updated.

Refresh and reconciliation

Refresh is not the same thing as reconciliation. Reconciliation is quite heavy-weight, trying to get the object from the resources, checking mappings and policies and so on. On the other hand, refresh is designed to a lightweight task. Refresh is going to execute operations that are easy and efficient, such as check whether shadow indexes are correct. When it comes to operations with the resources, refresh will only execute those operations that need to be executed. E.g. failed operations will be re-tried, but only if the re-try time was reached. Refresh is meant to set the shadow objects right, nothing more. It will not fix data on the resource, neither will it run the mappings to modify data of focal objects. The purpose of refresh is to make sure that shadow objects are right.

Refresh Task

Shadow refresh happens during normal operation of midPoint, e.g. it will happen during reconciliation process. However, sometimes it is preferable to execute just the refresh operation in a controlled manner. This may be achieved by shadow refresh task.

Shadow refresh task is a lightweight task that executes shadow refresh on selected shadows. Refresh task is very simple:

Refresh shadows

Git

<!--

  ~ Copyright (c) 2010-2024 Evolveum

  ~

  ~ Licensed under the Apache License, Version 2.0 (the "License");

  ~ you may not use this file except in compliance with the License.

  ~ You may obtain a copy of the License at

  ~

  ~     http://www.apache.org/licenses/LICENSE-2.0

  ~

  ~ Unless required by applicable law or agreed to in writing, software

  ~ distributed under the License is distributed on an "AS IS" BASIS,

  ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

  ~ See the License for the specific language governing permissions and

  ~ limitations under the License.

  -->

<task   xmlns="http://midpoint.evolveum.com/xml/ns/public/common/common-3"

        xmlns:q="http://prism.evolveum.com/xml/ns/public/query-3"

        xmlns:c="http://midpoint.evolveum.com/xml/ns/public/common/common-3"

        xmlns:org="http://midpoint.evolveum.com/xml/ns/public/common/org-3"

        oid="20bb3018-3169-403d-bdca-a20417fb70ee">

    <name>Shadow refresh</name>

    <assignment>

        <targetRef oid="00000000-0000-0000-0000-000000000528" relation="org:default" type="c:ArchetypeType">

            <!-- Utility task -->

        </targetRef>

    </assignment>

    <activity>

        <work>

            <shadowRefresh>

                <shadows>

                    <query>

                        <q:filter>

                            <q:all/>

                        </q:filter>

                    </query>

                </shadows>

            </shadowRefresh>

        </work>

    </activity>



    <schedule>

        <interval>10</interval>

        <recurrence>recurring</recurrence>

    </schedule>

    <executionState>suspended</executionState>

    <schedulingState>suspended</schedulingState>

</task>

Such refresh task will look for shadow objects that contain pending operations. The reason is that those are the objects that most likely need to be refreshed, e.g. failed operation has to be retried or the status of manual operation needs to be checked. This is a reasonable default for sustained operation of midPoint.

Explicit Filter

There are cases when refresh task needs to be run explicitly on a selected group of shadows (e.g. migration scenarios). In that case the search filter may be explicitly specified in the refresh task:

Refresh shadows with pending operations on specific resource

Git

<!--

  ~ Copyright (c) 2010-2024 Evolveum

  ~

  ~ Licensed under the Apache License, Version 2.0 (the "License");

  ~ you may not use this file except in compliance with the License.

  ~ You may obtain a copy of the License at

  ~

  ~     http://www.apache.org/licenses/LICENSE-2.0

  ~

  ~ Unless required by applicable law or agreed to in writing, software

  ~ distributed under the License is distributed on an "AS IS" BASIS,

  ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

  ~ See the License for the specific language governing permissions and

  ~ limitations under the License.

  -->



<task   xmlns="http://midpoint.evolveum.com/xml/ns/public/common/common-3"

        xmlns:q="http://prism.evolveum.com/xml/ns/public/query-3"

        xmlns:c="http://midpoint.evolveum.com/xml/ns/public/common/common-3"

        xmlns:org="http://midpoint.evolveum.com/xml/ns/public/common/org-3"

        oid="d19f9ce5-1cfa-40e6-a6e2-61c017630844">

    <name>Forced Shadow refresh</name>

    <assignment>

        <targetRef oid="00000000-0000-0000-0000-000000000528" relation="org:default" type="c:ArchetypeType">

            <!-- Utility task -->

        </targetRef>

    </assignment>

    <activity>

        <work>

            <shadowRefresh>

                <shadows>

                    <query>

                        <q:filter>

                            <q:text>resourceRef matches (oid = '10000000-0000-0000-0000-000000000004') and pendingOperation exists</q:text>

                        </q:filter>

                    </query>

                </shadows>

            </shadowRefresh>

        </work>

    </activity>

    <schedule>

        <recurrence>single</recurrence>

    </schedule>

    <executionState>suspended</executionState>

    <schedulingState>suspended</schedulingState>

</task>
Refresh shadows on specific resource (regardless of pending operations)

Git

<!--

  ~ Copyright (c) 2010-2024 Evolveum

  ~

  ~ Licensed under the Apache License, Version 2.0 (the "License");

  ~ you may not use this file except in compliance with the License.

  ~ You may obtain a copy of the License at

  ~

  ~     http://www.apache.org/licenses/LICENSE-2.0

  ~

  ~ Unless required by applicable law or agreed to in writing, software

  ~ distributed under the License is distributed on an "AS IS" BASIS,

  ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

  ~ See the License for the specific language governing permissions and

  ~ limitations under the License.

  -->



<task   xmlns="http://midpoint.evolveum.com/xml/ns/public/common/common-3"

        xmlns:q="http://prism.evolveum.com/xml/ns/public/query-3"

        xmlns:c="http://midpoint.evolveum.com/xml/ns/public/common/common-3"

        xmlns:org="http://midpoint.evolveum.com/xml/ns/public/common/org-3"

        oid="06997833-26a2-44c0-a6c4-cc079a88cb38">

    <name>Forced Shadow refresh</name>

    <activity>

        <work>

            <shadowRefresh>

                <shadows>

                    <query>

                        <q:filter>

                            <q:text>resourceRef matches (oid = '10000000-0000-0000-0000-000000000004')</q:text>

                        </q:filter>

                    </query>

                </shadows>

            </shadowRefresh>

        </work>

    </activity>



    <schedule>

        <recurrence>single</recurrence>

    </schedule>

    <executionState>suspended</executionState>

    <schedulingState>suspended</schedulingState>

</task>

Whenever you specify query, it overrides the default query (all shadows on all resources with pending operations).

Performance

Refresh task is quite lightweight and efficient. Therefore it can usually be scheduled for quite a frequent execution, usually executing every few minutes.

See Also

Was this page helpful?
YES NO
Thanks for your feedback