ConnId Connector Framework

Last modified 13 Apr 2021 17:52 +02:00

MidPoint is using ConnId framework to develop, manage and run identity connectors. This framework provides a layer that separates the identity management system from the target and source systems. Evolveum is a major contributor to the development of ConnId framework.

ConnId Framework Versions

There are, in theory, two major ConnId versions:

  • ConnId 1.x is real, working and practical connector framework. Existing connectors are written to work with ConnId 1.x. ConnId 1.x has some issues, mostly inherited from the past. However, it is a solid and practical choice for identity connectors. It is actively maintained, and it is part of several identity management products.

  • ConnId 2.x is just a plan now. The goal of ConnId 2.x is to correct the mistakes of original framework design. Tirasa and Evolveum, as two major contributors to ConnId framework have done numerous evolutionary improvements to ConnId 1.x. However, we are at the point where we need to make larger improvements. Improvements that will affect compatibility of existing connectors. Hence ConnId 2.x plan. The plan was agreed by ConnId community. However, due to various reasons this remains just a plan for now.

Framework and Connectors

The ConnId framework is developed jointly by several companies and independent contributors. However, the connectors have an independent life cycle. The connectors are developed and maintained in separate projects. There are mostly non-technical reasons for this separation such as licensing issues, philosophical differences and business strategies. However the common framework makes the connectors compatible. Therefore connectors from several projects can be used together in one solution.

For example, see the connectors that work with midPoint. Almost every connector is maintained in its own separate mini-project. Many of these connectors are developed by Evolveum. But there are connectors developed by Evolveum partners, contributed by the community, and there are connectors taken from ConnId project or even connectors that track their origin to OpenICF or Sun ICF.

The connectors depend on the framework, yet they can have a complete separate life cycle.

Connector Servers

Connectors can be deployed in two ways:

  • Local connectors are deployed to a midPoint instance. This is the usual way how connectors are used. The connector is executed inside a midPoint instance, has the same lifecycle (start/stop), etc. MidPoint can detect local connectors automatically and overall the connector management is easier.

  • Remote connectors are executed in a different process or on a different node than midPoint instance. Remote connectors are deployed to a connector server. There may be need to use a remote connector e.g. to access a file on a remote system (e.g. in case of CSV connector) or because of platform incompatibilities (e.g. .NET connectors)

Connector is not developed as local or remote. The placement of the connector is a deployment-time decision. There is just one connector package that can be deployed locally or remotely.

The Story of Identity Connectors as Seen by Evolveum

In the old days there were no identity connectors. Each IDM product used its own proprietary framework. Such early frameworks were usually ugly, dirty and cumbersome. A product called "Lighthouse" developed by a company called Waveset was no exception. The company was acquired by Sun Microsystems, and the product was renamed to Sun Identity Manager (Sun IDM). The engineers at Sun obviously realized how bad this "adapter" framework was, and after few long years of hesitation they finally created a new framework. It was still quite far from being perfect, but there was one huge difference: it was not proprietary. Sun developed the framework as an open source project. This project was known simply as "Identity Connector Framework" (ICF). Identity connectors were born. Before the ICF framework got any chance of major success, Sun was acquired by Oracle. We can only speculate what happened inside Oracle, but the result was that the ICF project effectively stopped all development activity. Last commit to the ICF project was in May 2010.

However, the acquisition of Sun was like a supernova. Engineers that worked with Sun technologies suddenly scattered around to other projects and companies. This also affected the team that now forms the core of midPoint project.

Back then, Oracle announced the end of Sun IDM development. Therefore we were looking for a replacement product. We have quickly realized that Oracle IDM or any similar commercial product just cannot satisfy our needs. We have decided to start a new open source project to fill this sudden technological gap. It was early 2010 when we connected with ForgeRock and started work on OpenIDM version 1. The Sun ICF framework was an obvious choice for a connector layer. Although we were not aware of it, another project was started approximately at the same time: Syncope. This project has also chosen ICF as a connector framework. In early 2011, ForgeRock decided to drop OpenIDM version 1 code-base. This was an impulse that contributed to our decision to start independent development of midPoint. The ICF was kept as a connector layer in a new midPoint project. So now there were three open source projects that were using the framework. This finally seems like a success for the framework. But there was a glitch.

In mid-2011 it was quite clear that the original Sun ICF project is not going anywhere. ForgeRock decided to take over the development and formed the OpenICF project. We have been forming an independent stream of development at that very time. Even though we were competing with ForgeRock, we had seen the benefit of cooperation. Therefore we have decided to cooperate on OpenICF. Approximately at the same time the ConnId project was created by the Syncope team. This was also a fork of the original Sun ICF code. There were also rumours that Oracle continues development of ICF in a closed-source fashion. Therefore in late 2011 there were actually several versions of ICF:

  • Original Sun Identity Connector Framework - in a clinical death state

  • OpenICF maintained by ForgeRock with Evolveum as a major contributor

  • ConnId maintained by the Syncope team

  • Oracle closed-source version (rumoured)

The "forks" began independent development, and it is perhaps no big surprise that the forks quickly became incompatible. This was quite an awkward situation. We could do nothing about the original Sun ICF, and it is unlikely that we could do anything about Oracle. However, having two incompatible open source frameworks was clearly insane. That was the time when our Project Polygon was born as an attempt to survive in this confusing situation. It took several years to make OpenICF and ConnId teams to talk to each other. But it finally happened in late 2013. The code of OpenICF was merged into ConnId. This worked quite well for some time. Yet later on, ForgeRock stopped contributing to ConnId. Without any official statement or notice to ConnId team, ForgeRock developed OpenICF framework independently. Other ConnId contributors were puzzled, but the development of ConnId went on. Then in 2016 ForgeRock stopped to publish their day-to-day development, effectively making OpenICF a closed-source project. However, Evolveum and Tirasa continued cooperation to maintain and extend ConnId framework, which continues to this day.

Currently there is one common framework code maintained in ConnId Project. The idea is to use this framework in all open source IDM projects (midPoint, Syncope and possibly others). Teams from Evolveum and Tirasa contribute the code to ConnId framework. ConnId connectors are compatible and interchangeable. All the teams also take part of the design and future development of the framework. We are more than aware that the ICF framework is not perfect. Therefore we have plans to improve it, in ConnId 2.0 framework. We take care to proceed in fully open and transparent fashion to make sure it does not become a proprietary technology.