Auditing

Last modified 24 Oct 2024 13:51 +02:00
Audit trail feature
This page describes configuration of Audit trail midPoint feature. Please see the feature page for more details.
Table of Contents

Auditing is a mechanism that records the most important interactions in the system in a computer-processable forms. The goal of the auditing is to record the interactions on "business" level, essentially recording who does what, who changed what, etc.

The audit record (audit trail) has to be machine-processable. It should be eventually possible to reconstruct a partial historical state of the system from the audit records by "going back in time".

Audit Record Structure

Field Abbreviation Description

Timestamp

The exact moment in time that the audit record was generated

Event Identifier

eid

Unique identification of an audit event. It is a Lightweight Identifier.

Event Type

et

Type of audit event. It describes what kind of operation was executed (adding an object, modification, user login, …​)

Event Stage

es

Stage of event processing: request of execution. See below.

Session Identifier

sid

Identifier of the interactive session in which the event have taken place. For security reason this is different than the content of the session cookie.

Task Identifier

tid

Unique identification of a task that this event is a part. It is a Lightweight Identifier.

Task OID

toid

OID of a task that this event is part of (if the task is persistent). Since 4.3 the OID of the task tree root is used, if applicable.

Host Identifier

hid

Identifier of a host that generated the audit record. This is the host name corresponding to a network interface that accepted the HTTP request.

Node Identifier

nid

Identifier of a node in a midPoint cluster that generated the audit record.

Remote Host Address

raddr

Network address of a remote host that originated the request causing this event.

Initiator

I

User that initiated ("caused") the event. It may the user that started the operation from a GUI or web service interface, the user that is owner of the task that recorded the event, etc.

Target

T

Object that is a primary target of an operation (if applicable). In case that operation targets more than one object, the "primary" or the most important is recorded (e.g. the user object)

Target Owner

TO

Owner of the object that is a target of this operation (if applicable). E.g. an owner of a task that is being stopped, owner of an account that is being modified, etc.

Deltas

D

The changes that are being made to the targets. Deltas contain detailed information about the operation.

Channel

ch

The channel that was the source of the operation that this record describes

Outcome

o

Result of the operation: success, failure, partial failure, etc.

Event Types

Event type defines the kind of operation that was executed. Currently, defined event types are shown in the table below.

Column Event DB ID is relevant only for the older Generic Repository. The new Native PostgreSQL Repository uses custom types displaying the name in the first column.
Event Type Event DB ID Description

GET_OBJECT

0

Unused

ADD_OBJECT

1

Creating a new object (e.g. adding a user, account, resource, …​)

MODIFY_OBJECT

2

Modification of an existing object

DELETE_OBJECT

3

Removing an object

EXECUTE_CHANGES_RAW

4

Executing changes directly in the repository, e.g. changes made via debug pages.

SYNCHRONIZATION

5

Synchronizing a change notification. This type should only be present in REQUEST stage, the EXECUTION stage of the event will provide a more specific event type.

CREATE_SESSION

6

Creating a new session, e.g. user login.

TERMINATE_SESSION

7

End of a session, e.g. user logout.

WORK_ITEM

8

Creation or completion of a work item (e.g. a request to approve particular assignment).

WORKFLOW_PROCESS_INSTANCE

9

Creation or completion of a workflow process instance (may include 0, 1 or many work items).

RECONCILIATION

10

Initiation or completion of reconciliation

SUSPEND_TASK

11

Suspending a task

RESUME_TASK

12

Resuming a task

RUN_TASK_IMMEDIATELY

13

Running a task immediately (now)

DISCOVER_OBJECT

14

Created shadow after discovering object on resource.
See Auditing of Resource Object Changes chapter.

Other event types may be added in the future (including custom event types).

Event Stages

Operations in midPoint may be subject to quite complex processing before they get executed. This may include processing RBAC, expressions, workflows or other "plugin" hooks. Therefore, the details of the operation may considerably differ at the beginning of the operation and at the end. Event stages are defined to denote this difference and to record events at various stages through the operation lifecycle.

Column Event DB ID is relevant only for the older Generic Repository. The new Native PostgreSQL Repository uses custom types displaying the name in the first column.
Event Stage Stage DB ID Description

REQUEST

0

The operation is requested. The record shows operation details (e.g. deltas) in the form as originally intended by the user or client.

EXECUTION

1

The operation after execution. The record shows operation details (e.g. deltas) in the form as it was executed.

RESOURCE

2

Stage which contains events that were recorded on provisioning level (objects on resources). Both before and after something happened in provisioning, e.g. when auditing operations executed through provisioning on managed resource as well as getting changes through live synchronization process.
See Auditing of Resource Object Changes chapter.

Other stages may be added in the future (e.g. stage for approvals or workflow steps).

There is usually one request, one execution and multiple provisioning records for each operation. E.g. the request record contains the delta that assigns role to a user. The execution record will also contain that delta, but it may contain additional deltas, e.g. deltas for adding new accounts implied by the role. In some cases there may be several execution records for a single request record. This happens if the execution happens in several waves. E.g. a role is assigned to a user. Some accounts implied by the role may be created immediately and others needs to wait for an approval. Therefore, the accounts that can be created immediately will be audited in a first execution audit record. The second batch of accounts will be audited when they are later approved and created. This goes to the second audit record. This situation may also happen even if there are no approvals, e.g. in case of resource dependencies or even complex inbound-outbound-template interactions.

Auditing of RESOURCE stage is enabled by default. This feature can be turned off in system configuration (systemConfiguration/audit/eventRecording/recordResourceStageChanges).

Event Outcomes

The result of the executed operation. All the possible values are described in the following table.

Column Event DB ID is relevant only for the older Generic Repository. The new Native PostgreSQL Repository uses custom types displaying the name in the first column.
Event Outcome Outcome DB ID Description

SUCCESS

0

Used when operation and sub operations finish successfully. The operation is completed and the result is final.

WARNING

1

Used when operation finish successfully, but minor problem occurred. For example operation code recovered from some error and after that operation finished successfully. The operation is completed and the result is final.

PARTIAL_ERROR

2

Used when operation contains at least one operation witch status SUCCESS/WARNING and at least one operation with status FATAL_ERROR. The operation is completed and the result is final.

FATAL_ERROR

3

Used when operation didn’t finish correctly. The operation is completed and the result is final.

NOT_APPLICABLE

4

Result does not make any sense for the operation. This is useful in cases that the operation is not supported (e.g. an optional part of the interface). This is different from UNKNOWN, as in this case we really know that its result is not applicable. In UNKNOWN case we know nothing. The operation is completed and the result is final.

IN_PROGRESS

5

The operation is being executed. This is set for operations that are executed asynchronously or take a significant amount of time. Short synchronous operations do not need to set this status, they may go well with the default UNKNOWN status. The operation is in progress and the final result is not yet known.

UNKNOWN

6

No information about operation is present. Presence of this status usually means programming bug, e.g. someone forgot to set or compute appropriate operation result.

HANDLED_ERROR

7

The operation didn’t finish correctly but that was expected and handled. It is equivalent to success for all practical cases except for displaying the result. But using success status for this situation might be misleading. The operation is completed and the result is final.

Initiator and Attorney

MidPoint 3.7 introduced a concept of attorney. Therefore, there is possibility that one user acts on behalf of another user. Both users are recorded in the audit logs. The meaning is as follows:

  • Initiator is the (legal) entity on behalf of whom is the action executed. It is the subject of the operation. Authorizations of the initiator are used to evaluate access to the operation. This is the entity who is formally responsible for the operation. Although initiator is always a user in midPoint 3.7 and earlier, the initiator may be an organization in later midPoint versions.

  • Attorney is the (physical) user who have executed the action. This is the user that have logged-in to the user interface. This is the user that pressed the button to execute the action. This is always identity of a user and it will always be a user. It cannot be a company or any other virtual entity.

Audit Trails

The auditing subsystem in midPoint is designed to be pluggable. There are currently two auditing implementations:

  • auditing to log files;

  • auditing to database tables.

To enable the audit implementation, auditService section must be added to the audit element of the config.xml. Inside the added auditService element, the implementation is chosen by the value of the auditServiceFactory element.

Logfile Auditing

Audit logs are recorded in a form of human-readable text records in the usual log files. This auditing goes to the default log file (idm.log) and is turned off by default. It is using a dedicated logger name:

Audit Logger Name
com.evolveum.midpoint.audit.log

This logger can be directed to a specific appender to a separate audit log file using the normal logging configuration mechanism.

To enable this logger:

  1. Add the following auditService element inside audit element:

    <auditService>
    <auditServiceFactoryClass>com.evolveum.midpoint.audit.impl.LoggerAuditServiceFactory</auditServiceFactoryClass>
</auditService>

  2. Enable it in the System Configuration under logging→auditing like this:

    <systemConfiguration>
    ...
    <logging>
        ...
        <auditing>
            <enabled>true</enabled>
            <details>false</details> <!-- true for DEBUG level with more details -->
            <appender>MIDPOINT_LOG</appender>
        </auditing>
    </logging>
</systemConfiguration>

Note that simply setting level for com.evolveum.midpoint.audit.log logger to INFO or DEBUG will NOT work, because the level is explicitly overridden based on the System configuration.

Database Table Auditing

Audit service writing to database tables is closely related to the repository implementation:

XDAS

Auditing implementation in midPoint was inspired by XDAS and it is conceptually compatible with XDAS. The actual XDAS support in midPoint is planned for the future.

XDAS is a specification of distributed auditing system developed by Open Group.

The XDAS specification asks for a common audit log format and a common taxonomy of audit log events.

The XDAS system is composed of several components. The components can be placed inside a single system or distributed across an organization.

Good introduction to XDAS architecture is provided by the OpenXDAS Project

Determining Remote Host Address

Normally, the remote host address is determined from the HTTP connection; as returned by the HttpServletRequest.getRemoteAddr() method. However, there are situations where a trustworthy proxy server is used, so the "real" client host address can be obtained from an HTTP header created by it. MidPoint can be set up to use such a header (if present) using the following configuration:

Reading client address from X-Forwarded-For HTTP header
<systemConfiguration>
  ...
  <infrastructure>
    <remoteHostAddressHeader>X-Forwarded-For</remoteHostAddressHeader>
  </infrastructure>
</systemConfiguration>

If there’s no such header, network-level client address is used.

If the header contains more values (separated by commas), the first i.e. leftmost one is used.

Resource oid column

EXPERIMENTAL

This feature is experimental. It means that it is not intended for production use. The feature is not finished. It is not stable. The implementation may contain bugs, the configuration may change at any moment without any warning and it may not work at all. Use at your own risk. This feature is not covered by midPoint support. In case that you are interested in supporting development of this feature, please consider purchasing midPoint Platform subscription.
Since 4.2
This functionality is available since version 4.2.

If we need work with resource oid in reporting, we can allow store resource oid for audit record to database. For that we need to add the next snippet of code to the System Configuration object.

<systemConfiguration>
    ...
    <audit>
        <eventRecording>
            <recordResourceOids>true</recordResourceOids>
        </eventRecording>
    </audit>
    ...
</systemConfiguration>

Native repository stores the information directly in ma_audit_event table in column resourceOid, see audit tables for more details. Generic repository uses m_audit_resource to store the information, see audit tables for more details.

Custom column

EXPERIMENTAL

This feature is experimental. It means that it is not intended for production use. The feature is not finished. It is not stable. The implementation may contain bugs, the configuration may change at any moment without any warning and it may not work at all. Use at your own risk. This feature is not covered by midPoint support. In case that you are interested in supporting development of this feature, please consider purchasing midPoint Platform subscription.
Since 4.2
This functionality is available since version 4.2.

When we need some other information in the audit table, we can store custom properties in additional custom columns in the database. This configuration is useful only when SQL audit trail is used.

Auditing of create/termination session event for channels rest and actuator

Since 4.2
This functionality is available since version 4.2.

Since version 4.2 channels for rest and actuator do not create audit records about session creation or termination by default. You can turn it on via variable in System Configuration audit→eventRecording→recordSessionlessAccess.

Separate repository configuration for audit

Since 4.2
This functionality is available since version 4.2.

By default, audit uses the same data source like the main repository. Since version 4.2 it is possible to set up audit in different database.

Auditing of Resource Object Changes

Since 4.8
This functionality is available since version 4.8.

Since version 4.8 midPoint audits also all operations performed on provisioning level - changes of resource objects. It means all provisioning operations, object discovery operations, …​

This feature enhances readability of the audit and displays some resource operations what were not yet audited (e.g. object discovery). For each resource operation midPoint adds new record to the audit with eventStage="resource".

Target name in the resource audit record is the name of the resource object, e.g. name of the account. So administrators can see which accounts were modified on the resources directly from the view.

Enabled by default

The feature is enabled by default. It can significantly increase audit log size, especially when the deployment manages a lot of resources.

If you need, you can disable auditing of resource object changes in SystemConfiguration.xml:

<systemConfiguration>
    ...
    <audit>
        <eventRecording>
            <recordResourceStageChanges>false</recordResourceStageChanges>
        </eventRecording>
    </audit>
    ...
</systemConfiguration>

Audit Trail Retention

Retention of audit trail records can be controlled by a cleanup policy. See Removing obsolete information for details.

Configuration examples

It is unlikely that only audit to a logfile is used, audit to a database is typically used as well. Because there are two different repository implementations, the examples are available on these pages:

See Also

Compliance

This feature is related to the following compliance frameworks:

Was this page helpful?
YES NO
Thanks for your feedback