MidPoint 4.3 "Faraday"

Last modified 11 May 2021 15:35 +02:00

Release 4.3 is a thirty-fifth midPoint release code-named Faraday. The 4.3 release brings performance improvements, user interface and diagnostic improvements and major internal improvements.

Release date09 April 2021
Release type Feature release
End of support09 April 2022
faraday.png

Michael Faraday (1791-1867) was an English scientist who contributed to the study of electromagnetism and electrochemistry. His research on electricity and magnetic fields lead him to inventions of electromagnetic rotary devices, which formed the foundation of electric motor technology. He was one of the most influential scientists in history, paving the way for practical use of electricity in technology. Even though it might not been obvious during his lifetime, Faraday's work changed our lives forever.

Similarly to Faraday's inventions, MidPoint 4.3 brings radical improvements. However, most of the improvements are still hidden under the hood. MidPoint 4.3 is paving the way for major leap in performance and scalability that is planned for midPoint 4.4. Yet, midPoint 4.3 still brings significant benefits on its own. There are numerous improvements to user interface and diagnostics, as well as performance improvements. MidPoint 4.3 is a significant step in the direction of large-scale midPoint deployments.

Changes With Respect To Version 4.2

New Features and Improvements

  • Major features

    • Preliminary results from MidScale project, bringing some performance and scalability improvements.

    • Human-friendly query language (experimental)

    • Significant improvement to password management page

    • Diagnostic and visibility improvements

    • Asynchronous (messaging) outbound resources (experimental)

  • User interface improvements

    • Flat list display for organizational units

    • Improved display of disabled roles

    • Sorting of values on preview page

    • Improved customization of basic/advanced search

    • Search through the entire role catalog (role request page)

    • Support for sorting of custom columns

    • Various minor usability and UX improvements

  • Miscellaneous improvements

    • Minor function library improvements

    • Support for expressions in certification definition search filters

    • Ability to filter/skip audit records by using custom expression

  • Diagnostics, Visibility, Robustness

    • Storing more operation errors in the objects

    • Retries of failed synchronization operations (experimental)

    • Improved handling of multi-node task status

    • Minor visibility improvements to triggers, expressions, task progress, …​

  • Internals and Development

    • Early prototype of scalable repository implementation based on PostgreSQL (experimental)

    • Code cleanups and dependency upgrades

    • Support for global namespace prefixes

    • Numerous smaller performance improvements in various places in the system.

    • Localization service passed to script evaluators

    • Significant improvements to Schrodinger testing framework

    • Support for storing objects in JSON in repository (experimental)

Deprecation, Feature Removal And Incompatible Changes

  • Use of HQL query language for audit log queries and dashboard widgets is deprecated since midPoint 4.2. Support of HQL for audit log queries and dashboard widgets is planned for removal in midPoint 4.4. Please use midPoint query language instead.

  • Use of Jasper-based reports in midPoint is deprecated since midPoint 4.2 in favor of the new "native" reports. The support for Jasper-based report is planned for removal in midPoint 4.4, therefore it is recommended to migrate the reports as soon as possible.

  • Support for specification of custom resource namespace (namespace item in ResourceType) is deprecated. The support will be dropped in midPoint 4.4.

  • Support for production deployments of midPoint in Microsoft Windows environment is deprecated. Microsoft Windows will still be supported for evaluation, demo, development and similar non-production purposes. Support for windows-based production deployments will be removed in midPoint 4.4.

  • Microsoft Internet Explorer is no longer supported. Support for Microsoft Internet Explorer was deprecated since midPoint 4.2.

  • MySQL and MariaDB are no longer supported. Support for MySQL and MariaDB was deprecated since midPoint 4.1. Please see Repository Database Support for details. It is strongly recommended to use PostgreSQL database instead.

  • JMX-based node-to-node communication in midPoint cluster is deprecated. Please use the default REST communication method instead.

  • Explicit deployment to an external web container is deprecated since midPoint 4.1.

  • MidPoint plug-in for Eclipse IDE was never officially supported and it will not be developed any more. This plugin is abandoned in favor of IntelliJ IDEA environment (MidPoint Studio).

Releases Of Other Components

  • New versions of LDAP Connector, Active Directory Connector and DatabaseTable Connector were released during the course of midPoint 4.3 development.

  • Docker images were released in Docker Hub: 4.3 and 4.3-alpine.

  • Overlay project examples were released together with midPoint 4.3 release.

  • MidPoint Studio version 4.3 was released shortly after midPoint 4.3 release. MidPoint Studio is officially a part of midPoint subscriptions now, covered by ordinary midPoint support programs.

  • Java REST client 4.3 will be released shortly after midPoint 4.3 release.

  • Schrodinger testing framework will be released shortly after Java REST client release.

Purpose and Quality

Release 4.3 (Faraday) is intended for full production use. It belongs to a feature release family, supported only for a reduced time period. Therefore it is intended for users that prefer new features over long-term stability.

All features are stable and well tested - except the features that are explicitly marked as experimental or partially implemented. Those features are supported only with special subscription contract.

Limitations

Following list provides summary of limitation of this midPoint release.

  • Functionality that is marked as Experimental Functionality is not supported for general use (yet). Such features are not covered by midPoint support. They are supported only for those subscribers that funded the development of this feature by the means of subscriptions and sponsoring or for those that explicitly negotiated such support in their support contracts.

  • MidPoint comes with bundled LDAP Connector. Support for LDAP connector is included in standard midPoint support service, but there are limitations. This "bundled" support only includes operations of LDAP connector that 100% compliant with LDAP standards. Any non-standard functionality is explicitly excluded from the bundled support. We strongly recommend to explicitly negotiate support for a specific LDAP server in your midPoint support contract. Otherwise only standard LDAP functionality is covered by the support. See LDAP Connector page for more details.

  • MidPoint comes with bundled Active Directory Connector (LDAP). Support for AD connector is included in standard midPoint support service, but there are limitations. Only some versions of Active Directory deployments are supported. Basic AD operations are supported, but advanced operations may not be supported at all. The connector does not claim to be feature-complete. See Active Directory Connector (LDAP) page for more details.

  • MidPoint user interface has flexible (fluid) design and it is able to adapt to various screen sizes, including screen sizes used by some mobile devices. However, midPoint administration interface is also quite complex and it would be very difficult to correctly support all midPoint functionality on very small screens. Therefore midPoint often works well on larger mobile devices (tablets) it is very likely to be problematic on small screens (mobile phones). Even though midPoint may work well on mobile devices, the support for small screens is not included in standard midPoint subscription. Partial support for small screens (e.g. only for self-service purposes) may be provided, but it has to be explicitly negotiated in a subscription contract.

  • There are several add-ons and extensions for midPoint that are not explicitly distributed with midPoint. This includes Java client library, various samples, scripts, connectors and other non-bundled items. Support for these non-bundled items is limited. Generally speaking those non-bundled items are supported only for platform subscribers and those that explicitly negotiated the support in their contract.

  • MidPoint contains a basic case management user interface. This part of midPoint user interface is not finished. The only supported part of this user interface is the part that is used to process requests and approvals. Other parts of case management user interface are considered to be experimental, especially the parts dealing with manual provisioning cases.

This list is just an overview, it may not be complete. Please see the documentation regarding detailed limitations of individual features.

Platforms

MidPoint is known to work well in the following deployment environment. The following list is list of tested platforms, i.e. platforms that midPoint team or reliable partners personally tested with this release. The version numbers in parentheses are the actual version numbers used for the tests.

It is very likely that midPoint will also work in similar environments. But only the versions specified below are supported as part of midPoint subscription and support programs - unless a different version is explicitly agreed in the contract.

Operating System

MidPoint is likely to work on any operating system that supports the Java platform. However, for production deployment, only some operating systems are supported:

  • Linux (x86_64)

  • Microsoft Windows Server (DEPRECATED, planned for removal in 4.4)

We are positive that MidPoint can be successfully installed on other operating systems, especially macOS and Microsoft Windows desktop. Such installations can be used to for evaluation, demonstration or development purposes. However, we do not support these operating systems for production environments. The tooling for production use is not maintained, such as various run control (start/stop) script, low-administration and migration tools, backup and recovery support and so on.

Java

  • OpenJDK 11 (11.0.10). This is a recommended platform.

OpenJDK 11 is a recommended Java platform to run midPoint.

Support for Oracle builds of JDK is provided only for the period in which Oracle provides public support (free updates) for their builds. As far as we are aware, free updates for Oracle JDK 11 are no longer available. Which means that Oracle JDK 11 is not supported for MidPoint any more. MidPoint is an open source project, and as such it relies on open source components. We cannot provide support for platform that do not have public updates as we would not have access to those updates and therefore we cannot reproduce and fix issues. Use of open source OpenJDK builds with public support is recommended instead of proprietary builds.

Web Containers

MidPoint is bundled with an embedded web container. This is the default and recommended deployment option. See Stand-Alone Deployment for more details.

Explicit deployment of war file to web container is deprecated. Following Apache Tomcat versions are supported:

  • Apache Tomcat 9.0 (9.0.37)

Apache Tomcat 8.0.x and 8.5.x are no longer supported. Support for explicit deployment to newer Tomcat versions is not planned. Please migrate to the default stand-alone deployment model as soon as possible.

Databases

MidPoint supports several databases. However, performance characteristics and even some implementation details can change from database to database. Since midPoint 4.0, PostgreSQL is the recommended database for midPoint deployments.

  • H2 (embedded). Supported only in embedded mode. Not supported for production deployments. Only the version specifically bundled with midPoint is supported.
    H2 is intended only for development, demo and similar use cases. It is not supported for any production use. Also, upgrade of deployments based on H2 database are not supported.

  • PostgreSQL 13, 12, 11 and 10. PostgreSQL 13 or 12 is strongly recommended option.

  • Oracle 12c

  • Microsoft SQL Server 2019, 2016 SP1

Our strategy is to officially support the latest stable version of PostgreSQL database (to the practically possible extent). PostgreSQL database is the only database with clear long-term support plan in midPoint. We make no commitments for future support of any other database engines. See Repository Database Support page for the details.

Only a direct connection from midPoint to the database engine is supported. Database and/or SQL proxies, database load balancers or any other devices (e.g. firewalls) that alter the communication are not supported.

Supported Browsers

  • Firefox

  • Safari

  • Chrome

  • Edge

  • Opera

Any recent version of the browsers is supported. That means any stable stock version of the browser released in the last two years. We formally support only stock, non-customized versions of the browsers without any extensions or other add-ons. According to the experience most extensions should work fine with midPoint. However, it is not possible to test midPoint with all of them and support all of them. Therefore, if you chose to use extensions or customize the browser in any non-standard way you are doing that on your own risk. We reserve the right not to support customized web browsers.

Important Bundled Components

Component Version Description

Tomcat

9.0.37

Web container

ConnId

1.5.0.17

ConnId Connector Framework

LDAP connector bundle

3.2

LDAP, Active Directory and eDirectory connector

CSV connector

2.4

Connector for CSV files

DatabaseTable connector

1.4.6.0

Connector for simple database tables

Upgrade

MidPoint is software that is designed for easy upgradeability. We do our best to maintain strong backward compatibility of midPoint data model, configuration and system behavior. However, midPoint is also very flexible and comprehensive software system with a very rich data model. It is not humanly possible to test all the potential upgrade paths and scenarios. Also some changes in midPoint behavior are inevitable to maintain midPoint development pace. Therefore we can assure reliable midPoint upgrades only for midPoint subscribers. This section provides overall overview of the changes and upgrade procedures. Although we try to our best, it is not possible to foresee all possible uses of midPoint. Therefore the information provided in this section are for information purposes only without any guarantees of completeness. In case of any doubts about upgrade or behavior changes please use services associated with midPoint subscription programs.

Upgrade From MidPoint 4.2.x

MidPoint 4.3 data model is not completely backwards compatible with previous midPoint versions. However, vast majority of data items is compatible. Therefore the usual upgrade mechanism can be used. There are some important changes to keep in mind:

  • Database schema needs to be upgraded using the usual mechanism. Please see MidPoint Upgrade Guide for details.

  • Version numbers of some bundled connectors have changed. Therefore connector references from the resource definitions that are using the bundled connectors need to be updated.

Upgrade From MidPoint 4.1.x Or Older

Upgrade from midPoint 4.1.x or older is not supported directly. Please upgrade to midPoint 4.2.x first.

Changes In Initial Objects Since 4.2

MidPoint has a built-in set of "initial objects" that it will automatically create in the database if they are not present. This includes vital objects for the system to be configured (e.g. role Superuser and user administrator). These objects may change in some midPoint releases. However, midPoint is conservative and avoids overwrite of customized configuration objects. Therefore midPoint does not overwrite existing objects when they are already in the database. This may result in upgrade problems if the existing object contains configuration that is no longer supported in a new version. The following list contains a summary of changes to the initial objects in this midPoint release. The complete new set of initial objects is in the config/initial-objects directory in both the source and binary distributions. Although any problems caused by the change in initial objects is unlikely to occur, the implementors are advised to review the changes and assess the impact on case-by-case basis:

  • 000-system-configuration.xml: added schedulingState to TaskType object details GUI configuration, added admin-dashboard configuration.

  • 021-archetype-system-role.xml, 022-archetype-business-role.xml, 521-archetype-task-approval.xml: Updated icons.

  • 040-role-enduser.xml: added lookup table get authorization.

Please review source code history for detailed list of changes.

Bundled Connector Changes Since 4.2

  • LDAP and AD connectors were upgraded to the latest available version 3.2.

  • DatabaseTable connector was upgraded to the latest available version 1.4.6.0.

Behavior Changes Since 4.2

  • Task OID in audit records now points to the root of the task tree, if applicable. Note that task identifier remains to be the identifier of actual task that executed the request.

  • Dead shadows remain linked to the focus (user). Handling links to dead shadows was inconsistent in previous midPoint versions. This was aligned in midPoint 4.3. Links to dead shadows are marked by relation of type "related".

  • Custom dashboards are not displayed automatically in the menu. Dashboards that are to be included in the menu have to be explicitly enabled in system configuration.

  • Requester information in notification handlers was corrected (MID-6754), which may be a minor compatibility issue.

Public Interface Changes Since 4.2

  • Prism API was changes in several places. However, this is not yet stable public interface therefore the changes are not tracked in details.

  • There were changes to the IDM Model Interface (Java). Please see source code history for details.

Important Internal Changes Since 4.2

These changes should not influence people that use midPoint "as is". These changes should also not influence the XML/JSON/YAML-based customizations or scripting expressions that rely just on the provided library classes. These changes will influence midPoint forks and deployments that are heavily customized using the Java components.

  • There were changes in internal code structure at numerous places do to refactoring and code cleanup. Most changes were related to the midScale effort. Heavy customizations of midPoint 4.2 may break in midPoint 4.3.

Known Issues and Limitations

As all real-world software midPoint 4.3 has some known issues. Full list of the issues is maintained in bug tracking system. As far as we know at the time of the release there was no known critical or security issue.

There is currently no plan to fix the known issues of midPoint 4.3 _en masse_. These issues will be fixed in future maintenance versions of midPoint only if the fix is covered by a support agreement or subscription. No other issues will be fixed - except for severe security issues that may be found in the future.

The known issues of midPoint 4.3 may or may not be fixed in following releases. This depends on the available time, issue severity and many variables that are currently difficult to predict. The only reliable way how to make sure that an issue is fixed is to purchase midPoint support. Or you can fix the bug yourself. MidPoint is always open to contributions.

This may seem a little bit harsh at a first sight. But there are very good reasons for this policy. And in fact it is no worse than what you get with most commercial software. We are just saying that with plain language instead of scrambling it into a legal mumbo-jumbo.

Some of the known issues are listed below:

  • There is a support to set up storage of credentials in either encrypted or hashed form. There is also unsupported and undocumented option to turn off credential storage. This option partially works, but there may be side effects and interactions. This option is not fully supported yet. Do not use it or use it only at your own risk. It is not included in any midPoint support agreement.

  • Native attribute with the name of 'id' cannot be currently used in midPoint (MID-3872). If the attribute name in the resource cannot be changed then the workaround is to force the use of legacy schema. In that case midPoint will use the legacy ConnId attribute names (icfs:name and icfs:uid).

  • We have seen issues upgrading H2 instances to a new version. Generally speaking H2 is not supported for any particular use. We try to make H2 work and we try to make it survive an upgrade, but there are occasional issues with H2 use and upgrade. Make sure that you backup your data in a generic format (XML/JSON/YAML) in regular intervals to avoid losing them. It is particularly important to backup your data before upgrades and when working with development version of midPoint.

Credits

Majority of the work on the Faraday release was done by the Evolveum team. However, this release would not be possible without the help of our partners, customers, contributors, friends and families. We would like to express our thanks to all the people that contributed to the midPoint project both by providing financial support, their own time or those that maintain a pleasant and creative environment for midPoint team. However, midPoint project would not exist without proper funding. Therefore we would like to express our deepest gratitude to all midPoint subscribers that made midPoint project possible.

Disclaimer

Planned release dates are just that: they are planned. We do not promise or guarantee release dates. Software development is a creative activity that includes a lot of inherent risk. We are trying really hard to provide the best estimates. We are not able to provide precise dates for releases or deliveries. Do not rely on midPoint release dates. Plan your project properly to address the risk of delayed midPoint releases.

Planned scope of midPoint releases is also an estimate. MidPoint development process always includes the balancing of the iron triangle. Therefore planned release scope may change at any time. There is a method to make sure that midPoint releases will work well for your project and that method is platform subscription.

We do not make any claims that midPoint is perfect. Quite the contrary. MidPoint is a practical software, developed by living and breathing developers and deployed in a real world. There are both known and unknown issues in midPoint. Also, midPoint is not feature-complete. New features are introduced in midPoint all the time. But not all of them are completed. There are always some limitations. As the license states, midPoint is provided "AS IS". Please do not rely on midPoint functionality that you have not tested to make sure that it works. MidPoint support and subscription programs are a way how to handle those issues. But even with support service, do not rely on functionality that is not documented. If you plan to use undocumented or non-existing functionality, platform subscription is the right service for you.