MidScale: MidPoint Scalability
MidPoint is an open source identity management and governance platform. MidPoint is an established solution for mid-size organizations providing transparency and accountability for personal data processing. Our midPrivacy initiative aims at implementation of unique data protection capabilities to midPoint, thus creating privacy-enhancing identity management platform. Overall, midPoint is a leading open source solution in several identity and access management areas.
However, midPoint has one significant limitation. MidPoint was originally built to address the needs of mid-size enterprises, agencies and universities. Initial design of midPoint data store components favored flexibility and time to market. As midPoint was targeting mid-size organizations the scalability was not high on a list of implementation priorities. But now, midPoint is being deployed to handle scenarios with large number of identities. Deployments that manage students, subscribers and consumers are becoming more and more common. Which makes sense, as these types of users can especially benefit from the data protection capabilities of midPoint. However, such deployments are hitting scalability limitations of current data storage components of midPoint.
Project Goals
Future scalability issues were foreseen in original midPoint design. MidPoint is not bound to any particular data store or database. Thanks to such foresight, midPoint has a flexible and replaceable data storage components. We would like to take advantage of this design feature and re-implement data storage components in a scalable way. Our plan is to redesign the database schema with scalability in mind. Also, we plan to take advantage of innovation in open source databases which was not readily available when midPoint development started almost a decade ago. Therefore we plan to fit our implementation specifically to PostgreSQL database to take full advantage of its capabilities.
We also plan to improve midPoint clustering mechanisms. Our goal is to support autoscaling capabilities used in cloud platforms, thus enhancing the on demand character of midPoint deployments. Higher scale also implies harder requirements on stability and robustness of the product. Therefore we plan to invest part of the effort to improve our quality assurance environment, especially focusing on scalability, performance and stability testing. MidPoint would greatly benefit from user experience improvements that can make administration of millions of identities easier.
Documents
-
Design and Architecture
-
Documentation
-
MidScale Infrastructure (coming soon)
-
Query Language (coming soon)
-
Blog, Articles And Other Media
-
Evolveum Blog
Timeline
Milestone | Goal | Planned date | Status |
---|---|---|---|
START |
Project start |
15-October-2020 |
DONE |
M1 |
Performance environment |
26 Nov 2020 |
DONE |
M2 |
Performance evaluation and repository analysis/design |
15 Jan 2021 |
DONE |
M3 |
Multithreading, Schrodinger and Query language |
26 Feb 2021 |
In progress |
M4 |
Performing Repo prototype, UI basic tests, Multinode Tasks |
12 Apr 2021 |
|
M5 |
PostgreSQL, Performance environment (2), UX Analysis & Design |
31 May 2021 |
|
M6 |
Repository optimization, performance evaluation |
9 Jul 2021 |
|
M7 |
Migration Procedure, GUI Improvements, Auto-scaling |
31 Aug 2021 |
|
FINISH |
Project finish |
14-October-2021 |
Funding
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the NGI_TRUST grant agreement no 825618.