Metrics

Last modified 27 Mar 2023 22:05 +02:00
Since 4.7
This functionality is available since version 4.7.

Metrics are numeric values attached to individual processed objects or their aggregations.

Currently, there are three types of metrics:

Table 1. Metrics types
Metric type Description Example Values Searchable Customizable

Built-in

Metrics that are automatically evaluated, without any need to define them.

added, deleted, modified

0 or 1

yes

no

Event mark-based

Bound to the presence of given event mark.

Focus activated, Projection renamed

0 or 1

yes

yes

Explicit

Explicitly defined by the engineer or administrator

number-of-modifications

any number (integer or non-integer)

no

yes

Notes:

  1. By "searchable" we mean whether it is possible to select processed objects with a specific value of given metric using midPoint query. Built-in and event mark-based metrics are searchable in this respect. Explicit ones are currently not. (However, they are still automatically fully processable in external tools after being exported in the form of a report.)

  2. By "customizable" we mean whether it is possible to change the definitions of individual metrics (event marks or explicit ones), as well as whether it is possible to enable or disable processing of these metrics for individual simulation runs. Built-in metrics cannot be customized; they are required for correct functioning of the midPoint GUI.

Built-in Metrics

There are the following built-in metrics. Each corresponds to a single value of SimulationResultProcessedObjectType.state property.

Table 2. Built-in metrics types
Metric Description State property value

added

1 if the object would be added, 0 otherwise

added

deleted

1 if the object would be deleted, 0 otherwise

deleted

modified

1 if the object would be modified, 0 otherwise

modified

When searching for an object with given built-in metric set to 1, the query on the state property should be used. (This metric is not recorded on individual processed objects explicitly.)

Event Mark-based Metrics

Each event mark present in the configuration is considered to be a simulation metric. Values of such metric are either 1 (mark is present) or 0 (mark is not present).

There is a set of default event marks present among midPoint initial objects. You can fully customize these, remove those that are not needed, and add your own ones.

Individual event marks can be enabled and disabled in a simulation result definition.

After simulation is run, it is possible to select individual processed objects carrying given event mark(s) by querying the SimulationResultProcessedObjectType.eventMarkRef item.

Explicit Metrics

It is possible to define any metrics over the set of processed objects.

For example, if we want to report on a number of modified resource object attributes, the following custom metric may be defined:

Listing 1. Custom metric example
<metric>
    <identifier>attribute-modifications</identifier>
    <computation>
        <domain>
            <expression>
                <script>
                    <code>processedObject.shadow</code> (1)
                </script>
            </expression>
        </domain>
        <valueExpression>
            <script>
                <code>processedObject.attributeModificationsCount</code> (2)
            </script>
        </valueExpression>
    </computation>
</metric>
1 Tells midPoint to apply this metric only to shadows (resource object).
2 Computes the value by calling built-in getAttributeModificationsCount() method on ProcessedObject instance. This method counts the number of attribute modifications that would be applied to the given resource object. Please see the Java documentation and/or the source code of ProcessedObject class to learn more.

Metric Definition

The metric definition has the following items:

Table 3. Metric definition items
Item Description Default

identifier

Identifier of the metric.

required

description

User-level description of the metric.

-

documentation

Technical documentation of the metric.

-

display

Display style of the metric.

-

displayOrder

An order in which this metric should be displayed relative to other metrics within the same context.

at the end

enabledByDefault

Is this metric enabled by default?

true

computation

How is the metric value computed for individual "processed objects"?

required, unless this metric is computed from a different (source) object-level metric

aggregation

How are the metric values aggregated?

plain summation

The computation element has the following items:

Table 4. Metric computation definition items
Item Description Default

domain

Domain of the metric. It denotes objects that should be considered by the metric.

all objects

selection

A predicate that determines whether the object is "selected" by the metric.

if valueExpression is present: true` if computed value is greater than 0, false otherwise

valueExpression

An expression providing the value for the metric. (See below for detailed description.)

if selection criteria are present: 1 if the object matches the criteria, 0 otherwise

Metric Domain and Selected Objects

There are two relevant object sets for each metric:

  1. Domain of the metric. These are all objects on which it makes sense to compute the metric value. Any objects outside the domain are simply ignored by the metric.

  2. Objects selected by the metric. The default interpretation is that these objects have above-zero value of the metric. However, you can modify the selection criteria to create custom aggregations over simple metrics, like "the number of VIP users with more than 5 modifications".

Metric Value Expression

The valueExpression is any midPoint expression - typically, a Groovy script - that computes the value of the metric for given object.

There are the following input variables:

Table 5. Value expression input variables
Variable Description

processedObject

Instance of ProcessedObject type.

modelElementContext

Instance of ModelElementContext object after the (simulated) processing. Normally, it does not need to be used. But it can provide some extra information if needed.

Please see the Java documentation and/or the source code of ProcessedObject class to learn more about what information it can provide and how to access it.

The expression should return a numeric value - e.g. integer, float, double, up to BigDecimal.

If it returns nothing, the value of the selection predicate is used to determine the result. If the selection predicate is not defined or returns nothing as well, the object is considered to be out of the domain of this metric.

Simulation Objects Predicates

MidPoint allows to specify metric domain and selection (and their further restrictions) by using so-called simulation object predicates.

A predicate contains any or all of the following:

Table 6. Simulation object predicate components
Item Description

filter

A filter over SimulationResultProcessedObjectType objects that denotes the matching ones.

expression

An expression evaluated on ProcessedObject instances that denotes the matching ones.

An example of the filter-based predicate that is used to specify the domain of a metric as the set of all users:

Listing 2. Specifying domain using filter
<domain>
    <filter>
        <q:equal>
            <q:path>type</q:path>
            <q:value>c:UserType</q:value>
        </q:equal>
    </filter>
</domain>
Listing 3. Specifying domain using expression
<domain>
    <expression>
        <script>
            <code>
                import com.evolveum.midpoint.xml.ns._public.common.common_3.UserType
                processedObject.isOfType(UserType.class)
            </code>
        </script>
    </expression>
</domain>

Currently, there is only a single variable available:

Table 7. Selection expression input variables
Variable Description

processedObject

Instance of ProcessedObject type.

Metric Values Aggregation

Values computed for individual processed objects are aggregated into a form that can be presented for the simulation result as a whole. The default aggregation is a plain summation: the resulting value is a sum of values for individual objects. However, there are other aggregation functions. They refer to the following basic elements.

Table 8. Elements for aggregation functions
Element Description

selectionSize

Number of objects selected by this metric.

selectionTotalValue

Sum of metric values for all objects selected by this metric.

domainSize

Number of objects in the domain of this metric.

domainTotalValue

Sum of metric values for all objects in the domain of this metric.

And the aggregation functions are:

Table 9. Aggregation functions
Function Value is computed as

none

The metric should not be aggregated.

selectionSize

selectionSize

selectionTotalValue

selectionTotalValue

domainSize

domainSize

domainTotalValue

domainTotalValue

domainTotalValueToDomainSize

domainTotalValue / domainSize

selectionTotalValueToDomainSize

selectionTotalValue / domainSize

selectionSizeToDomainSize

selectionSize / domainSize

selectionTotalValueToDomainTotalValue

selectionTotalValue / domainTotalValue

domainMinValue

minimal metric value in the domain

selectionMinValue

minimal metric value in the selection

domainMaxValue

maximum metric value in the domain

selectionMaxValue

maximum metric value in the selection

Definition of Metric Aggregation

Table 10. Metric aggregation definition items
Item Description Default

aggregationFunction

An aggregation function used to compute the (aggregated) metric value.

selectionTotalValue

source

Source metric that is to be aggregated. The metric must be present on individual processed objects, i.e. it is not possible to aggregate the aggregation-only metric.

The metric being defined, i.e. by default, we are defining an aggregation for the current metric being defined.

domainRestriction

Restriction of the domain of original metric. We may focus the aggregation on a subset of original objects.

original domain is not changed

selectionRestriction

Restriction of the object selection of original metric.

original selection is not changed

Examples

This is how we would compute the average number of attribute modifications per single shadow seen by the simulation activity. We refer to the attribute-modifications metric defined in above.

Listing 4. Computation of average number of attribute modifications per account seen
<metric>
    <identifier>avg-modifications-per-account-seen</identifier>
    <aggregation>
        <aggregationFunction>domainTotalValueToDomainSize</aggregationFunction>
        <source>
            <identifier>attribute-modifications</identifier>
        </source>
    </aggregation>
</metric>

However, we might be interested in the average number of modifications per shadows that were modified. We may now utilize the fact that shadows modified are, in fact, the objects selected by the source attribute-modifications metric.

Listing 5. Computation of average number of attribute modifications per account modified
<metric>
    <identifier>avg-modifications-per-account-modified</identifier>
    <aggregation>
        <aggregationFunction>domainTotalValueToSelectionSize</aggregationFunction>
        <source>
            <identifier>attribute-modifications</identifier>
        </source>
    </aggregation>
</metric>

Imagine now that we are interested in total number of changes in accounts in 'Security services' department. We can restrict the selection (or domain) of the original metric to cover only selected shadows.

Listing 6. Computation of total number of attribute modifications in Security services department
<metric>
    <identifier>modifications-of-security-services-accounts</identifier>
    <aggregation>
        <aggregationFunction>selectionTotalValue</aggregationFunction>
        <source>
            <identifier>attribute-modifications</identifier>
        </source>
        <selectionRestriction>
            <filter>
                <q:equal>
                    <q:path>after/attributes/ri:department</q:path> (1)
                    <q:value>Security services</q:value>
                </q:equal>
            </filter>
        </selectionRestriction>
    </aggregation>
</metric>
1 The after item represents the object state after the operation. (Note that if the department attribute itself is modified to a value other than "Security services", this change is not seen by this simple filter.)
Was this page helpful?
YES NO
Thanks for your feedback