ICF Issues

MidPoint is using ConnId connector framework as a basic framework to support identity connectors. ConnId project is a continuation of Identity Connector Framework (ICF) project that was originally started by Sun Microsystems. After the Oracle-Sun acquisition the open source Identity Connector Framework was abandoned. The code was later picked up by the community in a form of ConnId project. ConnId project is now maintaining and improving the framework. Evolveum is one of the principal participants in ConnId project.

The original ICF is burdened with numerous problems. Some of those problems were already addressed during the course of ConnId project development. But there is a legacy of problems that go very deep into the original ICF design - problems that are difficult, if not entirely impossible to fix. This page was created to list such problems.

Even though ICF (and therefore also ConnId) is burdened with problems, we have still chosen to use it. The number of existing connectors, tool chain and a relatively widespread knowledge of connector development still outweighs the problems of the framework. In addition to that we are working hard to improve ConnId framework and we are also trying to work around some of the ICF design issues inside midPoint. ConnId still remains fully supported and recommended way to create identity connectors for midPoint.

The Identity Connector Framework (ICF) design seems to heavily burdened with Sun IDM compatibility. The interface design is sub-optimal for the purpose of integrating quite a disparate set of connectors. There are also problems with the fundamental approach. From my blog (summer 2010):

The mandatory __NAME__ and UID is a bad idea. TODO: explain

There is no way how to share parsed schema between connector instances and no way how to persist/cache it. E.g. LDAP schema is quite bit. Now every connector instance needs to get LDAP schema and parse it. It is enough for just one instance to do it and then share it with all the instances that have the same configuration. The schema also changes very infrequently. It would be nice to store (or cache) it persistently.

TODO: maybe we need a concept of resource where the schema can be cached and persisted …​ as is in midPoint?

According to JavaDoc, Uid may be local or global. Obviously local means that it is unique only within an object class, global means that it is unique across all object classes. The framework does not gives us any mechanism how to tell whether UID is global or local.

Improvement: it would but nice to have ability to limit schema processing only to selected object classes. E.g. Active Directory schema is huge, but we do not need vast majority of the object classes. They just sit there and consume memory. It would be nice to only limit the processing to the object classes that we really need.

The ICF has a nice sync(..) operation that calls back a handler with a delta. This is supposed to poll for changes on the connector side.

On the SPI side the read and search operations are considered to be the same (both using SearchOp interface). This is wrong. Reading an object by using a primary identifier is almost always much more efficient than searching for it and it very ofter requires entirely different operation. Therefore most of the non-trivial connectors fist try to analyze the search filter, check if it is a search by identifiers and branch the code in a strange and slightly unreliable manner.

Also, for some resources it is much more efficient to read an object if we know that at most one object can be returned. As read is perhaps the most frequent connector operation this is unquestionably worth to be optimized.

Another issue: There are objects that have huge number of attribute values. E.g. big LDAP/AD groups. Some servers have mechanism to return only part of the values. This is what is needed in searches: the clients wants the list of objects. The client does not care about 100% completeness as it usually only displays a table of the results or uses the result to iterate over the objects. On the other hand, when the client explicitly gets one specific object the completeness is crucial. In ICF design it is not really possible to distinguish these cases.

The concept of UID is entirely naive. It assumes that every object will have a persistent, immutable, non-complex identifier. This not always the case. Therefore the UID is defined as sometime immutable but in fact mutable. This also assumes that the object has a single identifier which is both immutable and also the most efficient way to retrieve an object. This is not the case e.g. for LDAP containers. The ICF identfies the container by UID (see OperationOptions). The UID is usually entryUuid in LDAP. But the search operation requires DN as a base, not entryUuid. Therefore ICF forces the connector to make two searches instead of one: translate UID to DN and then do the actual search.

All the attributes are equal but UID is more equal than others. UID is supposed to be a primary identifier and it is passed to many operations as an explicit argument (update, getObject, delete, …​). But not for create. In that case the UID is just an ordinary attribute.

Both UID and NAME attributes are mandatory and both are regarded as identifiers. However, there is a lot of systems that have only one identifier that is typically mutable: username. The ICF does not handle well this kind of systems.

ICF designers obviously realized that by changing one attribute other attribute may change as well. That’s the reason why update methods return new version of Uid. So fat so good. But Uid is not the only attribute for which this works. This is hardcoded in the interfaces. This lack of foresight is unbelievable …​

Use case: LDAP servers change the value of naming attribute when the DN changes. E.g. if DN changes from cn=foo,dc=example,dc=com to cn=bar,dc=example,dc=com, the LDAP server also changes value of cn attribute from foo to bar. ICF Uid is not changes (that is usually entryUUID LDAP attribute). The connector has no way how to let the IDM know that the cn has changed. Yet, the change of naming attribute is often quite consequential …​

The ICF framework does not define any(!) excpetion. All the provisioning, communication and schema exceptions are supposed to be run-time exceptions, they are neither enumerated nor documented. The description of the error is frequently buried inside a labyrinth of nested exceptions. Therefore a very ugly and fragile code is needed in ICF client to determine cause of the problem.

Even worse, ICF is throwing exceptions that are not visible outside connector implementation. The connectors run in their own classloaders. The framework throws the exceptions almost in the same form as thrown by the connectors. Therefore, exception from a library loaded by the connector classloader may be exposed from the ICF API. The ICF client cannot see that exception as it does not have access to the exception class. Therefore attempts to serialize and deserialize such exception (which is common in Web GUI error handling pages) will fail. ICF should never expose objects that are outside the interface contract and are not visible to the client.

Bundles are versioned, but connectors are not. It does not make sense to bind versioning to a package instead of the code itself.

It is not clear whether connector configuration schema is specific to a connector version or connector type. It does not really make sense to be either. In the former case upgrades will be difficult, in the later version the configuration cannot be evolved. Obviously a major-minor version mechanism is needed.

Wrapper type that overrides toString so the sensitive value will not accidentally appear in the logs or dumps is a good idea. Similarly the dispose() method that will shorten the memory lifetime of the sensitive value. However, encrypting the value with key store right next to it is plainly a pointless exercise that only complicates the system and is a potential source of bugs.

Those result handlers are an artifact of an original Identity Connector Framework over-engineering. The handlers are supposed to assist connectors by implementing mechanism that the connector or resource does not support - such as search result filtering, data normalization and so on. However, those handler are generic and they know nothing about the particulars of the resource that the connector connects to. Therefore in vast majority of cases those handlers just get into the way and they distort the data. Good connectors usually do not need those handlers at all. Unfortunately, these handler are enabled by default and there is no way for a connector to tell the framework to turn them off. The handlers needs to be explicitly disabled in the resource configuration.

ICF has no real concept of capabilities. Connector can demonstrate the capabilities indirectly. E.g. delete capability is demonstrated by implementing SPI class DeleteOp. However, this does not reflect runtime status. E.g. connector may be capable of delete operation, but the resource that the connector is connected to may not support that operation. There is no way how connector can indicate that.

Some capabilities may also have complex parameters. E.g. update operation may need to know a complete state of the account to be able to update it. There is no way to indicate that. There is also no way how to indicate supported scripting languages for script operations.

Some capabilities are determined from the schema, e.g. enable/disable capability. There is an open question whether this is the right way to do it. E.g. disable operation may have side-effects, e.g. destruction of user’s password. There is not way to indicate that. Also the password capability can be detected from the schema. But there is no way how to indicate password policy.

The service account is configured specifically for each connector using a connector-specific configuration. It is not in any way structured or annotated. Therefore it does not allow some of the features, e.g.:

A better way would be for the configuration to point to the IDM service account instead of just hard-configuring the credentials.

To be politically correct, org.identityconnectors.common.logging.Log.Level is one big mess. It makes absolutely no sense for a connector to log on INFO or SEVERE log levels. The connector should not change system state in a significant way (INFO), or place system in a SEVERE situation. On the other hand, there is no way how to distinguish debugging and tracing logs.

Configuration properties and attributes are handled in entirely different way. E.g. configuration properties use String[] class to denote multi-valued properties while attributes use Flags.MULTIVALUED. The ConfigurationProperty has isConfidential() method while Attribute has nothing like that and the confidentiality needs to be derived from the use of Guarded types.

TODO