MidPrivacy: Data Provenance Prototype

Data provenance is one of major problems of data protection and identity management in general. The problem may be summarized by a simple question "Where did the data come from?". This is a simple question, but the answer is surprisingly complex. Were the data provided by the user? When did the user provide that data? For what purpose? Were the data retrieved from the HR system? Are they bound to a specific contract? Were the data received from a third party? Were they created as a part of "social login" or a membership in identity federation? Was it a combination of several systems? Do we have conflicting data coming from different sources?

The question of data provenance is a critical problem for transparency and accountability. Data controllers and processors often lose track of data origin. But how can one implement proper data protection if data origin is not known? It would be almost impossible to demonstrate how a particular data item was obtained, that it was handled properly and that there is existing legal basis for processing of that particular item. Smaller and simpler organization can probably track this using a "paper processes". However, this approach is not feasible for larger and complex organizations. Data protection policies and processes must be automated. The problem is, that this requires to create a metadata model – we need to maintain a complex metadata about each data item of the (primary) data model. E.g. for each value of user’s e-mail address we need to know origin, creation dates, modification dates, expiration dates, processing purposes or reference to specification a lawful basis and possibly a lot of other information. An additional problem is that this meta-data set is likely to evolve in time as it needs to adapt to privacy policies and evolution of data protection regulations.

Outcome of Data Provenance phase of midPrivacy project is prototype implementation of privacy-enhancing features in midPoint. This phase of the project was aimed at prototyping and evaluating data provenance capabilities. Existing midPoint data model functionality was enhanced with the metadata schema capabilities to track origin (provenance) of every data item. Such changes required development of new data modelling language: Axiom. The project included development of prototype user interface aimed at intuitive presentation of data provenance to a user. Overall goal of the enhancement is it improve transparency and accountability of personal data processing.

Project outcomes:

Functionality developed in the scope of this project is part of midPoint 4.2 release. As this is a prototyping project, the functionality is not considered to be production-ready and it is formally considered to be experimental.

Please see outcomes document for full details about project outcomes.