Simulation

Last modified 13 Mar 2023 21:06 +01:00

Since 4.7

This functionality is available since version 4.7.

Introduction

The "simulation" is an umbrella term covering various mechanisms of "what-if" analysis in midPoint. It could be characterized as an ability to see expected effects of some actions without the risk of damaging the system state.

Let us introduce few basic terms first: execution mode and configuration used.

Execution mode determines whether any actions carried out by an activity will have persistent effects or not.
Configuration used tells whether the activity should use "production" or "development-mode" configuration.

Figure 1. Execution modes and system configurations

Let us describe these concepts in more details.

Persistent-Effects vs Simulation Execution Mode

Any activity in midPoint can execute in two basic modes: persistent-effects or simulation.

Persistent-Effects Mode

This is the standard mode of operation, where any actions are really executed, be that in midPoint repository or on the resources. All such changes are also recorded in the system audit log.

There are some specialties like postponing resource operations either because of the resource unavailability (planned - see maintenance mode - or unplanned), or because of provisioning propagation. But none of these contradict the basic idea that all computed operations are going to be (eventually) applied.

This mode can be sometimes (imprecisely) called production. But this term conflicts with the one of Production Configuration, so it should not be used.

Overall, the "persistent-effects mode" term is not ideal either. We are looking for a better one.

Simulation Mode

Here, no actions that could have persistent externally-visible effects are executed. Accordingly, nothing is written to the system audit log.

The changes are computed, just like in Persistent-Effects Mode but instead of their application, they are written to so-called simulation result. It is a special kind of objects in midPoint repository, collecting both statistical and detailed information about changes that would be applied.

Production vs Development Configuration

Production Configuration

The production configuration comprises all configuration items that are engaged in regular midPoint operations. By configuration item we mean for example a definition of resource, resource object type, attribute, association, abstract role, assignment, or mapping.

Items in the production configuration are the only ones that can cause persistent effects. (See Persistent-Effects Mode.)

Development Configuration

Development configuration represents the part of the system configuration that is currently being developed. With regard to the production configuration, some items may be added, some replaced, and finally, some may be missing.

The development configuration cannot be executed in "persistent-effects" mode.

Before 4.7, the development configuration had to reside in separate midPoint instance called typically "development" or "testing" one. Although such approach still has its advantages, now it is possible that both configurations share a single midPoint instance.

Distinguishing Production and Development Configurations

This distinction is currently done using the lifecycleState property on configuration items.

Figure 2. Lifecycle states and configurations

Table 1. Mapping of item’s lifecycle state to configuration set membership
Lifecycle state	Production Configuration	Development Configuration
`active` or missing	yes	yes
`proposed`	no	yes
`deprecated`	yes	no
other	no	no

Lifecycle state is currently supported on the following configuration items:

resource,
resource object class,
resource object type,
resource object attribute,
resource object association,
abstract roles (role, org, service, archetype),
assignment,
object template,
mapping.

For short, we will sometime use the statements "configuration item (like resource) is in development mode" to denote the fact that the item is part of the development configuration but not present in the production one. On the other hand, when saying "item is in production mode", we mean that it is a part of the production configuration, with not saying anything whether it is also part of the development configuration.

Examples

Resource in Development Configuration Only

The following resource is in development configuration only.

Listing 1: Example of a resource in development configuration only

<resource xmlns="http://midpoint.evolveum.com/xml/ns/public/common/common-3">
    <name>CSV resource</name>
    <!-- ... -->
    <lifecycleState>proposed</lifecycleState>
    <!-- ... -->
</resource>

This setting guarantees that the content of the real resource will not be touched by midPoint. All operations that could affect this resource could be only simulated.

Replacing a Mapping in an Object Template

The following setup can be used to preview a migration from old to new organizational unit by switching mappings in an object template.

Listing 2: Example of switching mappings in an object template

<objectTemplate xmlns="http://midpoint.evolveum.com/xml/ns/public/common/common-3">
    <name>person-template</name>
    <mapping> (1)
        <strength>strong</strength>
        <expression>
            <value>ACME</value>
        </expression>
        <target>
            <path>organization</path>
        </target>
    </mapping>
    <mapping>
        <lifecycleState>deprecated</lifecycleState> (2)
        <strength>strong</strength>
        <expression>
            <value>old-unit</value>
        </expression>
        <target>
            <path>organizationalUnit</path>
        </target>
    </mapping>
    <mapping>
        <lifecycleState>proposed</lifecycleState> (3)
        <strength>strong</strength>
        <expression>
            <value>new-unit</value>
        </expression>
        <target>
            <path>organizationalUnit</path>
        </target>
    </mapping>
</objectTemplate>

1	No explicit lifecycle state means "active". Hence, this mapping is part of both production and development configurations.
2	The `deprecated` state means that this mapping belongs only to the production configuration.
3	The `proposed` state means that this mapping belongs only to the development configuration.

The third mapping serves as a replacement of the second one in the development configuration. Hence, when you run a persistent-effect activity (or a simulation one using production configuration), users are assigned to old-unit. But when running a simulation using development configuration, the users are assigned to new-unit instead.

Typical Simulation Scenarios

This section presents a couple of typical simulation-related scenarios.

Please see the tutorial where these scenarios are shown in more details.

Introduction of a New Resource Configuration

When connecting new source or target system to midPoint, we must create its resource definition. However, the first version of the definition is rarely completely correct and usually there is a need to fine-tune it. MidPoint supports this by allowing us to evolve the definition safely, in development mode, so that potential mistakes will not cause any harm in production data in midPoint nor on resources.

Processes directly supported in this area:

classification of resource objects (e.g. accounts) into object types, like account/default or account/person;
correlation of resource objects to so-called focus objects, like users, roles, and so on;
mapping of data between resource objects and focus objects.

Evolution of a Resource Configuration

Eventually, the first version of resource configuration is completed and put into production use. Sooner or later, requirements for its evolution emerge. The simulations feature supports such evolution by allowing adding pieces of the configuration (like new object types, attributes, association, or mappings) first in the development mode, and only after being developed and tested enough, switching them into production. Also, other items can be replaced, by marking them as deprecated and adding analogous proposed items. Yet other items can be deprecated without replacement, effectively planning for their removal.

Specific Functionality

Resource Wizard

TODO resources and other configuration items are in "proposed" state by default
TODO switching between "proposed", "active", and "deprecated" states
TODO previewing content of a resource in simulation mode, so that re-classification occurs instantly

TODO implement + document

Running the Simulation on Foreground

TODO @Vilo?

Running the Simulation on Background

The simulations are typically executed in background tasks that host individual activities. There are three aspects that can be defined:

Execution Mode

An activity can be run in one of three basic modes:

Full execution. All computed changes are applied, everything has persistent effects. This is Persistent-Effects Mode described in the introduction.
Simulation (or preview). Changes to the data (user properties, account attributes, and so on) are only simulated, nothing is really changed in midPoint nor on resources. This is Simulation Mode described in the introduction.^[1]
Shadow management simulation (or preview). This is a special low-level simulation of shadow-management-related changes that are normally applied automatically. Those changes are related to shadow classification i.e. determining its kind and intent, and shadow correlation i.e. determining its owner. In modes 1 and 2 such changes are applied automatically, because they have no visible effects on the identity data. However, we may preview even those changes, to make development of classification and correlation configuration easier. This is an experimental functionality. Please see the tutorial for an example.

This configuration is driven by execution/mode parameter in the activity definition.

For more information, please see activity execution mode specification.

Configuration Used

An activity can execute against Production Configuration or Development Configuration. (The latter cannot be used in Persistent-Effects Mode.)

This configuration is driven by execution/configurationToUse parameter in the activity definition.

For more information, please see activity execution mode specification.

Simulation Result

We can run the simulation with or without creating the simulation result. Furthermore, there are various options connected with the result itself, for example, what metrics and event marks should we include in the result.

This configuration is driven by reporting/simulationResult parameter in the activity definition.

For more information, please see Configuring Simulation Results and activity simulation result definition.

An Example

Listing 3. An import task executing in simulation mode, over development configuration, and with creation of a simulation result

<task xmlns="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
      xmlns:ri="http://midpoint.evolveum.com/xml/ns/public/resource/instance-3"
      oid="8b169df3-3124-4e36-871f-83bb52acfd7b">
    <name>Simulated import from HR</name>
    <executionState>runnable</executionState>
    <activity>
        <work>
            <import>
                <resourceObjects>
                    <resourceRef oid="236dd5ca-47df-403c-82e1-9ce2f36be000"/>
                    <objectclass>ri:AccountObjectClass</objectclass>
                </resourceObjects>
            </import>
        </work>
        <execution>
            <mode>preview</mode> (1)
            <configurationToUse>
                <set>development</set> (2)
            </configurationToUse>
        </execution>
        <reporting>
            <simulationResult/> (3)
        </reporting>
    </activity>
</task>

1	The value of `preview` means we do not want to execute the actions in this task.
2	`development` means we want to use the development configuration in this task.
3	The presence of `simulationResult` means we want to create the simulation result object.

Displaying and Managing the Simulation Results

TODO @Vilo?

Configuring Simulation Results

Simulation results are configured in the system configuration and in individual simulation activities. Aspects that are configured include:

What metrics and event marks should be observed during the particular simulation?
Should a dedicated database partition be used for storing this simulation result?

Please see Simulation Result Definition for more information.

Reporting

Simulation results can be displayed interactively via midPoint GUI, or their content can be exported using the reporting functionality.

There are the following built-in simulation reports.

Table 2. Built-in simulation reports
Report	Description	Typical use
Results	Lists simulation results without having a look at individual processed objects. Shows metadata and individual metrics.	overview, trends
Objects	Lists objects processed by given simulation. One processed object corresponds to one report row.	high-level view of one simulation
Objects with Metrics	Lists objects processed by given simulation along with metrics related to each object.	more detailed quantitative analysis of the simulation
Items Changed	Lists items changed within individual processed objects.	changes of items, suitable especially for single-valued ones
Values Changed	Lists values added or deleted within individual items in processed objects.	changes of items, suitable especially for multivalued ones with larger number of values

Application Programming Interface

TODO

Configuring Lifecycle State

TODO

Limitations

TODO

1. Simulation and preview are used as synonyms.