"Name";"Administrative status";"Valid from";"Nick";"AssignmentOid";"Subtype"
"testUser01";"enabled";"2020-07-07T00:00:00.000+02:00";"nick1";"00000000-0000-0000-0000-000000000008,00000000-0000-0000-0000-000000000004";"sub1,sub22"
"testUser02";"enabled";"2020-07-07T00:00:00.000+02:00";"NICK2";;Import From Report
|
Advanced Feature
The import from report, often requires scripting and other special configuration options. If the set-up is not correct you are risking potential override or deletion of midPoint and/or account data. Proceed with caution. |
Import Behaviour 'Import From report'
MidPoint supports 'import reports'/'reverse reports'. This feature is still experimental. The basic idea is that the report output generated by midPoint can also be used in reverse way - you can import it back to midPoint. In addition, it is also possible to import a custom defined report output.
This might be handy if you wand to update or change some already imported objects in some other way than through the midPoint UI (e.g. in some spreadsheet) and reimport them back. Yet as mentioned above, this feature is still experimental and such details often require custom configuration.
Midpoint support two kind of import configuration. Configuration for Object Import and Import script.
Object Import
MidPoint has to understand the report output data structure to preform the import correctly. This is configured in the report (ReportType), in a similar way as for exporting, so the configuration has to contain the information about columns and paths to which they are mapped. For details on how to configure standard reports please see this link. For now, only reports generated from object collections and CSV format is supported (CSV FileFormatType).
Example of Imported File:
CSV file
Example below shows report (ReportType) configuration for importing CSV file with header and two records above.
Import Behavior Example
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Copyright (c) 2024 Evolveum
~
~ Licensed under the Apache License, Version 2.0 (the "License");
~ you may not use this file except in compliance with the License.
~ You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<report>
<name>Object Collection import report with view</name>
<objectCollection>
<view>
<column>
<name>nameColumnCollection</name>
<path>name</path>
<display>
<label>Name (Collection)</label>
</display>
</column>
<column>
<name>activationColumn</name>
<path>activation/administrativeStatus</path>
<previousColumn>nameColumnCollection</previousColumn>
</column>
<column>
<name>validFromColumn</name>
<path>activation/validFrom</path>
<previousColumn>activationColumn</previousColumn>
</column>
<column>
<name>nickColumn</name>
<path>nickName</path>
<display>
<label>Nick</label>
</display>
<previousColumn>validFromColumn</previousColumn>
<import>
<expression>
<script>
<code>import com.evolveum.midpoint.prism.polystring.PolyString
return new PolyString("New nick: " + input)
</code>
</script>
</expression>
</import>
</column>
<column>
<name>assignmentColumn</name>
<path>assignment</path>
<display>
<label>AssignmentOid</label>
</display>
<previousColumn>nickColumn</previousColumn>
<import>
<expression>
<script>
<code>
import com.evolveum.midpoint.xml.ns._public.common.common_3.AssignmentType;
import com.evolveum.midpoint.xml.ns._public.common.common_3.ObjectReferenceType;
import com.evolveum.midpoint.xml.ns._public.common.common_3.RoleType;
assignments = new ArrayList();
for (String oid : input) {
if (oid != null) {
role = new ObjectReferenceType();
role.setOid(oid);
role.setType(RoleType.COMPLEX_TYPE);
AssignmentType assignment = new AssignmentType();
assignment.asPrismContainerValue()
assignment.setTargetRef(role);
assignments.add(assignment)
}
}
return assignments
</code>
</script>
</expression>
</import>
</column>
<column>
<name>subtypeColumn</name>
<path>subtype</path>
<previousColumn>assignmentColumn</previousColumn>
</column>
<type>UserType</type>
</view>
</objectCollection>
<behavior>
<direction>import</direction>
<!-- In case of non raw execution -->
<!-- <importOptions>-->
<!-- <modelExecutionOptions>-->
<!-- <raw>false</raw>-->
<!-- </modelExecutionOptions>-->
<!-- </importOptions>-->
</behavior>
</report>Behaviour and Options
You have to specify the direction of the report,in this case import and rather than the export direction, for this you need define the element behavior. Behavior contains the direction Import or Export. Also, a part of the behavior configuration is the parameter importOptions, which has the following elements:
| Name | Description | Type |
|---|---|---|
overwrite |
If set to "true", objects that are in the repository will be overwritten by the imported objects. It may not be applicable to all import types. E.g. it makes no sense for import from resource, as this is not storing objects in the repository directly. |
boolean |
keepOid |
If set to "true", objects that are overwritten will reuse the same OID as previous objects. May be potentially dangerous. USE WITH CARE. |
boolean |
stopAfterErrors |
Number of errors that will cause import to stop. If set to one the import will stop on first error. If set to zero or a negative value the import will continue in case of any number of errors. |
int |
summarizeSucceses |
If set to true the successfully imported items will be summarized in the result. WARNING: setting this to false may result in a very large result structure and may cause overflow of the system memory. |
boolean |
summarizeErrors |
If set to true the import errors will be summarized in the result. |
boolean |
referentialIntegrity |
boolean |
|
validateStaticSchema |
boolean |
|
validateDynamicSchema |
boolean |
|
encryptProtectedValues |
boolean |
|
fetchResourceSchema |
boolean |
|
keepMetadata |
If set to true then the metadata from the source file will be kept after the operation. If set to false then the original metadata will be replaced with new metadata on each object. |
boolean |
modelExecutionOptions |
If present, these options are used for adding objects into the repository. Null option values might be overridden by import-related options. In particular, the missing "raw" option is overridden to "true". So, if you want the operation run in non-raw mode, set "raw" option to "false" (e.g. runs also global templates, policy configuration, etc…). |
ModelExecuteOptionsType |
compatMode |
Compatibility model. If selected then the data parsing will be less strict. E.g. removed element will be ignored. |
boolean |
In previous example of report we define mapping values from columns to items in new object. The name of the column in the CSV file has to be same as the name defined in view. Definition of the name from a view has some rules. Name is obtained from Label of DispalyType for column, when Label is empty, then Midpoint finds the name from the item definition based on the Path element in a column.
Definition of a column also contains import/expression which can be used to define a script for generating items. Scripts have to return a real value, for example String or List of values for multivalued items, for example List<AssignmentType>. The script has a variable input which is a String when the item is single valued or a List<String>, when item is multivalued.
Import Script
You can define importScript in element behaviour. The Script contains variables with the same name as headers of imported CSV file. For example the following file will provide us with the variables with names username, role_name, action, valid_from and valid_to.
Example of Imported File
CSV file
"username";"role_name";"action";"valid_from";"valid_to"
"testUser02";"Superuser";"A";"2018-01-01";"2025-05-01"
"testUser01";"Superuser";"D";;
"fakeUser";"Superuser";"M";"2018-01-01";"2025-05-01"
"jack";"Superuser";A;"2018-01-01";"2025-02-01"
"jack";"Superuser";"M";"2018-01-01";"2025-05-01"
"jack";"FakeRole";"M";"2018-01-01";"2025-05-01"In the next example we change the assignments of users based on username.(e.g. the first line 'testUser02'). The operation which is executed is specified by the variable action ('A'=add, 'M'=modify, 'R'=remove). The target of the assignment is chosen via the variable role_name, which represents a role in midPoint. Variables valid_from and valid_to are mapped to the properties activation/validFrom and activation/validTo of assignment.
Import Script Behavior Example
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Copyright (c) 2024 Evolveum
~
~ Licensed under the Apache License, Version 2.0 (the "License");
~ you may not use this file except in compliance with the License.
~ You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<report>
<name>Report with import script</name>
<behavior>
<importScript xmlns:c="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
xmlns:s="http://midpoint.evolveum.com/xml/ns/public/model/scripting-3">
<s:options>
<s:continueOnAnyError>true</s:continueOnAnyError>
</s:options>
<s:pipeline>
<s:search xmlns:q="http://prism.evolveum.com/xml/ns/public/query-3">
<s:type>UserType</s:type>
<s:searchFilter>
<q:equal>
<q:path>name</q:path>
<c:expression>
<c:script>
<c:code>username</c:code>
</c:script>
</c:expression>
</q:equal>
</s:searchFilter>
</s:search>
<s:execute>
<s:forWholeInput>true</s:forWholeInput>
<s:script>
<s:code>
if (input == null || input.getData().isEmpty()){
log.error("Couldn't find user with name" + username + ". Skip this line.")
}
</s:code>
</s:script>
</s:execute>
<s:modify>
<s:parameter>
<s:name>delta</s:name>
<s:execute>
<s:parameter>
<s:name>outputItem</s:name>
<c:value>ObjectDeltaType</c:value>
</s:parameter>
<s:script>
<s:code>
import com.evolveum.midpoint.xml.ns._public.common.common_3.AssignmentType;
import com.evolveum.midpoint.xml.ns._public.common.common_3.ObjectReferenceType;
import com.evolveum.midpoint.xml.ns._public.common.common_3.RoleType;
import com.evolveum.midpoint.xml.ns._public.common.common_3.UserType;
import com.evolveum.midpoint.xml.ns._public.common.common_3.ActivationType;
import java.text.SimpleDateFormat;
import java.util.GregorianCalendar;
import com.evolveum.midpoint.prism.equivalence.EquivalenceStrategy;
import javax.xml.datatype.DatatypeFactory;
import com.evolveum.midpoint.schema.DeltaConvertor;
log.info("-----------START-----------");
log.info("username: " + username);
log.info("role_name: " + role_name);
log.info("action: " + action);
log.info("valid_from: " + valid_from);
log.info("valid_to: " + valid_to);
log.info("input: " + input);
user = input;
userBefore = user.clone();
role = midpoint.searchObjectByName(RoleType.class, role_name);
if (role == null) {
log.error("Couldn't find role with name " + role_name);
return null;
}
if (action.equals("A")) {
roleRef = new ObjectReferenceType();
roleRef.setOid(role.getOid());
roleRef.setType(RoleType.COMPLEX_TYPE);
AssignmentType assignment = new AssignmentType();
assignment.setTargetRef(roleRef);
if (valid_from != null || valid_to != null) {
activation = new ActivationType();
format = new SimpleDateFormat("yyyy-MM-dd");
if (valid_from != null) {
date = format.parse(valid_from);
cal = new GregorianCalendar();
cal.setTime(date);
xmlGregCal = DatatypeFactory.newInstance().newXMLGregorianCalendar(cal);
activation.setValidFrom(xmlGregCal);
}
if (valid_to != null) {
date = format.parse(valid_to);
cal = new GregorianCalendar();
cal.setTime(date);
xmlGregCal = DatatypeFactory.newInstance().newXMLGregorianCalendar(cal);
activation.setValidTo(xmlGregCal);
}
assignment.setActivation(activation);
}
user.getAssignment().add(assignment);
} else if (action.equals("M")) {
for (AssignmentType assignment : user.getAssignment()) {
if (assignment.getTargetRef() != null && role.getOid().equals(assignment.getTargetRef().getOid())) {
if (valid_from != null || valid_to != null) {
activation = new ActivationType();
format = new SimpleDateFormat("yyyy-MM-dd");
if (valid_from != null) {
date = format.parse(valid_from);
cal = new GregorianCalendar();
cal.setTime(date);
xmlGregCal = DatatypeFactory.newInstance().newXMLGregorianCalendar(cal);
activation.setValidFrom(xmlGregCal);
}
if (valid_to != null) {
date = format.parse(valid_to);
cal = new GregorianCalendar();
cal.setTime(date);
xmlGregCal = DatatypeFactory.newInstance().newXMLGregorianCalendar(cal);
activation.setValidTo(xmlGregCal);
}
assignment.setActivation(activation);
}
break;
} else {
log.warn("Couldn't find role with name " + role_name + " in the users assignments for modification.");
}
}
} else if (action.equals("D")) {
for (AssignmentType assignment : user.getAssignment()) {
if (assignment.getTargetRef() != null && role.getOid().equals(assignment.getTargetRef().getOid())) {
user.getAssignment().remove(assignment);
break;
} else {
log.warn("Couldn't find role with name " + role_name + " in the users assignments for deletion.");
}
}
} else {
log.error("Action column have unexpected value '" + action + "'")
return null;
}
if (userBefore.equals(user)) {
log.error("Couldn't create delta, because user before executing of script is same as after executing of script.")
return null;
}
delta = userBefore.asPrismObject().diff(user.asPrismObject(), EquivalenceStrategy.IGNORE_METADATA);
log.info("delta: " + delta);
log.info("-----------FINISH-----------");
return DeltaConvertor.toObjectDeltaType(delta);
</s:code>
</s:script>
</s:execute>
</s:parameter>
</s:modify>
</s:pipeline>
</importScript>
<direction>import</direction>
</behavior>
</report>See Also
Was this page helpful?
YES
NO
Thanks for your feedback