ConnId Connector Framework

ConnId framework is the intermediary layer between connectors and the IDM system (client). Framework is responsible for managing the connectors. The framework loads connector code. It initializes the connector and maintains pool of connector instances in ready-to-use state. The framework routes the operations to appropriate connector.

Framework does not know specific details about the system that the connectors connect to. The framework has no idea about LDAP, it does not know how to use your REST API, it does not even really care about the attributes and values of your accounts. The framework is just an intermediary, its responsibility is to manage the connectors and pass the information. The framework is designed to be generic and universal. All the details about the protocols used to connector to your system and the data structures that the system uses are hidden in the connector.

Despite that, the framework sometimes tries to be somehow intelligent. For example, it can simulate some operations that the connector cannot perform. However, in practice, these features often cause more problems than they solve. It is generally recommended to keep the framework agnostic and keep all the system-specific details in the connector.

The basic concept of ConnId is a connector. The connector is a piece of (usually) Java code that is governed by a uniform Java interface on one side (ConnId SPI) and that is using a variety of protocols and interfaces to connect to the resource on the other side.

Connector is essentially just a protocol translator. The responsibility of a connector is to get a command from the ConnId SPI and execute it using any protocol which is appropriate for the connected resource (SQL, LDAP, REST, etc.)

For example, LDAP protocol will react to the invocation of the create(…​) operation by sending LDAP add request to the LDAP server. This request will contain data about a new account that the connector translates from the ConnId representation (Java collection of Attribute objects) to LDAP representation (LDAP entry).

The connector knows all the system specific details. It knows how the LDAP protocol works, which operations are needed to create an LDAP entry, how the entry is structured, what LDAP attributes are mandatory and which are optional, what LDAP error codes means and so on. However, the connector usually has very little "business" logic. Even the little logic that the connector has will only deal with handling of the communication protocol and protocol data formats. Except for a few special-purpose attributes, the connector usually does not understand the meaning of the data that are passed through it. Vast majority of the IDM logic is implemented in identity management systems, such as midPoint. E.g. midPoint mappings copies and transforms attribute values, midPoint handles resource capabilities, synchronization, etc. The connector does not need to deal with that. The connector just translates protocols and data formats.

Connectors are distributed in bundles which is an ConnId packaging mechanism. A bundle can contain several connectors. However, most connector bundles contain only a single connector. Connector bundles are versioned, and the ConnId framework allows to run a connector in several versions at the same time.

In the old days there were no identity connectors. Each IDM product used its own proprietary framework. Such early frameworks were usually ugly, dirty and cumbersome. A product called "Lighthouse" developed by a company called Waveset was no exception. The company was acquired by Sun Microsystems, and the product was renamed to Sun Identity Manager (Sun IDM). The engineers at Sun obviously realized how bad this "adapter" framework was, and after few long years of hesitation they finally created a new framework. It was still quite far from being perfect, but there was one huge difference: it was not proprietary. Sun developed the framework as an open source project. This project was known simply as "Identity Connector Framework" (ICF). Identity connectors were born. Before the ICF framework got any chance of major success, Sun was acquired by Oracle. We can only speculate what happened inside Oracle, but the result was that the ICF project effectively stopped all development activity. Last commit to the ICF project was in May 2010.

However, the acquisition of Sun was like a supernova. Engineers that worked with Sun technologies suddenly scattered around to other projects and companies. This also affected the team that now forms the core of midPoint project.

Back then, Oracle announced the end of Sun IDM development. Therefore we were looking for a replacement product. We have quickly realized that Oracle IDM or any similar commercial product just cannot satisfy our needs. We have decided to start a new open source project to fill this sudden technological gap. It was early 2010 when we connected with ForgeRock and started work on OpenIDM version 1. The Sun ICF framework was an obvious choice for a connector layer. Although we were not aware of it, another project was started approximately at the same time: Syncope. This project has also chosen ICF as a connector framework. In early 2011, ForgeRock decided to drop OpenIDM version 1 code-base. This was an impulse that contributed to our decision to start independent development of midPoint. The ICF was kept as a connector layer in a new midPoint project. So now there were three open source projects that were using the framework. This finally seems like a success for the framework. But there was a glitch.

In mid-2011 it was quite clear that the original Sun ICF project is not going anywhere. ForgeRock decided to take over the development and formed the OpenICF project. We have been forming an independent stream of development at that very time. Even though we were competing with ForgeRock, we had seen the benefit of cooperation. Therefore we have decided to cooperate on OpenICF. Approximately at the same time the ConnId project was created by the Syncope team. This was also a fork of the original Sun ICF code. There were also rumours that Oracle continues development of ICF in a closed-source fashion. Therefore in late 2011 there were actually several versions of ICF:

The "forks" began independent development, and it is perhaps no big surprise that the forks quickly became incompatible. This was quite an awkward situation. We could do nothing about the original Sun ICF, and it is unlikely that we could do anything about Oracle. However, having two incompatible open source frameworks was clearly insane. That was the time when our Project Polygon was born as an attempt to survive in this confusing situation. It took several years to make OpenICF and ConnId teams to talk to each other. But it finally happened in late 2013. The code of OpenICF was merged into ConnId. This worked quite well for some time. Yet later on, ForgeRock stopped contributing to ConnId. Without any official statement or notice to ConnId team, ForgeRock developed OpenICF framework independently. Other ConnId contributors were puzzled, but the development of ConnId went on. Then in 2016 ForgeRock stopped to publish their day-to-day development, effectively making OpenICF a closed-source project. However, Evolveum and Tirasa continued cooperation to maintain and extend ConnId framework, which continues to this day.

Currently there is one common framework code maintained in ConnId Project. The idea is to use this framework in all open source IDM projects (midPoint, Syncope and possibly others). Teams from Evolveum and Tirasa contribute the code to ConnId framework. ConnId connectors are compatible and interchangeable. All the teams also take part of the design and future development of the framework. We are more than aware that the ICF framework is not perfect. Therefore we have plans to improve it, in ConnId 2.0 framework. We take care to proceed in fully open and transparent fashion to make sure it does not become a proprietary technology.