MidConnect

Inbound data transfer components (a.k.a. "enrollment components") are components that take care of processing user data when a new user lands in our system. The components will try to determine whether we can get user’s data using some of the supported "identity protocols" such as SAML, OpenID Connect or OAuth2.

The key part of the solution is the component which we now call "flexible authentication". This component implements the client side of the "identity protocols" (SAML is the only currently supported protocol). The original purpose of this component is to authenticate user for access to midPoint administration user interface. This component will need to be slightly re-structured and extended during this project, as we are interested in more than mere authentication. The component will need to determine user’s data, e.g. from SAML attribute assertions send in the request.

It is expected that the new user entering the system will need to be routed through an interactive "enrollment" procedure. This may be though of as an elaborate and semi-automated user registration procedure. Due to the data protection rules, we need to process user’s data on a specific basis. The basis specifies what data items are required, which are optional and so on. The user has to be informed about the use of personal data, and there may be a need for user’s consent to use optional data items (see below). The interaction might perhaps be skipped, in case that the sending party (identity provider) has already cleared the data requirements and choices with the user and there is a sufficient proof that it happened. However, even in that case, we may need to ask the user to fill in missing data items that the sending party have not provided. The exact details of this procedure are not yet entirely clear, as understanding of all applicable data protection regulations is not easy. Filling in the details may need to use legal services in the area of data protection to get relevant advice. However, we are almost certain that user interaction will need to take place at least in some cases.

The components may also need to correlate the identity with a database of existing identities in a system. This is necessary to avoid undesirable duplication of identities. It is also necessary for user convenience, e.g. we do not want to ask user for data that the user has already provided in previous interactions. However, identity correlation is not an easy task and there seem to be no single and simple solution. It is likely that user interaction will be required, e.g. user confirming the correlation match, even proving the identity. We plan to re-use identity correlation capabilities that midPoint already has. However, those capabilities were designed for enterprise environment, therefore the existing correlation functionality is non-interactive. Extension of those capabilities need to be considered.

It is essential the enrollment procedure - and in fact the entire personal data processing - follows the proper data protection procedures. We can only process those data items that are necessary to carry out the service, or data items that the user explicitly allowed to use. In other words, we must always process the data according to the basis for data processing.

Basis for data processing is a fundamental concept of personal data protection. We have encoutered this concept before and identified it as one of the most important concepts for identity management. The basis applies also in the "enrollment" case.

The fundamental insight is that enrollment/registration never happen in a vacuum. There is always some basis for registration, e.g. user becoming community member, registration for interest group, etc.

The basis specifies data requirement. The data requirement is a light of data items that are necessary or desirable for the purpose that the basis represents. As long as the basis applies, we can store the specified data items and use them for the specific purpose. When the basis no longer applies, we have to erase the data, unless there is another basis that applies to the data. The basis effectively controls the lifecycle of data items.

In midPoint, the basis can be represented by an abstract role, which is usually a Role, Org or a Service. This seems to be a very good fit, which is also partially confirmed by the experiments in "phase 0" of midPrivacy initiative. The results of provenance also support this choice, as was described in Provenance, Origin and Basis document.

While the basis specifies data requirement, there are likely to be optional data items that the user may choose to release or choose to keep private. Therefore, the basis itself is not enough just by itself, we need a place where to record the choices of individual users. We need a place where we can record user’s data release information. The data release needs to be bound to a particular basis, it may be time-bound, and there will certainly be metadata (e.g. date and time of the release). MidPoint assignment data structure seems to be a suitable place to store data release information. Assignment is naturally bound to abstract role that represents the basis. There are also metadata and time constraints. Although the choice to use assignment to record data release information has to be validated, it looks like a good choice at this stage of the project.

The situation is schematically illustrated in the following diagram:

Traditionally, identity management systems worked with directory servers and databases in a single organization. The IDM systems usually assumed that the data are always available, that the data can be fetched directly from the source. This a fair assumption to make, it also adds a benefit of having a fresh data, which allows early detection and propagation of data changes. However, the situation is quite different when it comes to the cross-organizational case.

When the original source of the data is in different organization then the destination of the data, we cannot assume that we have instant access to the data source. This is amplified by the nature of current "identity protocols" such as SAML, where the data are in fact available only at the moment of user authentication. This means that midPoint will need to store or cache the data, to have them available as needed. MidPoint has a concept of shadow objects that represent the data in a remote system. The shadow usually contains just the most essential identifiers of the remote objects. However, there is an experimental functionality to cache all the data in midPoint, which can be re-used in this project.

In the usual intra-organizational case, midPoint stores the compiled authoritative data in the user object. In that case there is very little motivation to cache data in the shadow. However, in intra-organizational case there is usually a very small number of data sources such as HR system or company directory. These source are known a priori, and the data flow are governed by organizational-wide policies that are mostly static. Therefore the authoritative, compiled state of the user object can be easily computed as needed. However, the cross-organizational case is vastly different. There are diverse data sources, such as identity providers in a federation. While the federations, social networks, community sites and similar data sources cannot appear in an entirely arbitrary way, the situation is considerably much more dynamic than it is in a typical intra-enterprise case. However, the most significant difference is that the data flow is not controlled solely by the policies. Choices of the users are the ultimate determinant in allowing or prohibiting a data flow. Additionally, the same identity is likely to be present in several data sources (several social network or federations), or even in several instances of the same data source (several identity providers in the same federation). Individual data sources may have inconsistent or even conflicting data, therefore simple policy-based mapping of the data to the user object will not work. Caching the original data in their original "raw" form may help resolve the conflicts, especially in case that user interaction will be required. In fact, the shadow may work like a "lightweight persona", enabling various approaches to merging identity data. The exact mechanisms is still unknown, but we are reasonably certain that any viable mechanism will need the "raw" data from the shadow.

In addition to the data caching capability, the shadow creates a data link to the original source of the data. Therefore the data could, theoretically, be refreshed when needed. However, there are usually protocol limitations. E.g. SAML protocol makes the data available only during (interactive) authentication flow. Yet, there may be some chances, such as using OpenID Connect refresh token, or combining the primary protocols with a more CRUD-like secondary protocols (such as SCIM). This is an avenue that is worth exploring in the project, although it is perhaps too early to commit to any tangible results.

MidPoint has a comprehensive and feature-rich user interface. This user interface is designed for advanced users. It can be used to manage the identities, configure and monitor midPoint platform. However, the inherent complexity of such user interface is a limiting factor for ordinary users. There is small "end user" part of the existing user interface that is designed for ordinary users. However, we are constantly receiving feedback from users that the user interface is complex and non-intuitive. We have made several attempts to improve the end user part of the interface. However, as the interface is technologically and conceptually part of much larger functionality, it is very hard to make any dramatic change in overall approach. We have concluded that the existing midPoint administrative user interface is not suitable for end users.

We believe that long-term data maintenance is an essential for data portability. Transfer of data from one system to another is just a first step in the data portability flow. An ability to maintain the data is at least as important as the initial data transfer. As there are numerous limitations of existing protocols, complexities of identity correlation, data protection regulations and other factors, long-term maintenance of personal data will required fair amount of user interaction.

As existing MidPoint administrative user interface is not a promising development path for interaction with end users, we have decided to try a different approach. We have decided to start working on a new user interface, targeted specifically at end users, with a strong focus on user experience. The ultimate goal is to create an user interface that an ordinary user can understand and use in an intuitive way. There are likely to be some challenges, as we have to deal with complex concepts of data origin, transfer, linking and data protection.

Technologically, we plan to start the work from an existing user interface of Perun, an identity management system that Masaryk University uses to manage identities. We would like to reuse the technology and some of the concepts, which is supported by participation of some members of Masaryk University development team. Important part of the development effort is a strong focus on user experience (UX). The implementation will begin with requirements analysis and preliminary definition of the scope. Some required functionality is obvious (such as modification of some user data), other functionality is not (extend of credential management, management of privileges, groups, etc.) The scope has to be carefully chosen to avoid over-complicating the interface. This will be accompanied by a work on UX aspects. We plan to develop the interface in iterations, gathering feedback from users and applying user experience improvements during the development. The goal is to have simple, but usable interface at the end, rather than complex and unusable one.

The self-service user interface will connect to midPoint server by the means of midPoint RESTful interface. Almost all the services needed by the user interface should be already available in the existing RESTful interface of midPoint. We plan to add the missing services or parameters as needed.