Ninja

Last modified 10 Oct 2023 09:53 +02:00
Table of Contents

Since midPoint 4.8. Also in 4.4.6, 4.7.2 and later.

  • Commands and options names realigned:

    • All command and option names are kebab cased

    • File output is now -o, --output

    • -O, --overwrite for all commands

Ninja is a command line tool bundled with midPoint. Implemented features are based on "repo-ninja", but code was rewritten from the ground up. This tool comes in handy when one needs to export or import objects from/to midPoint. Operations can be done on top of repository (DB) layer or through REST API. Access through DB layer can be very powerful, especially in cases when UI fails. For example when midPoint fails to start because of configuration issue or objects data corruption. Another great example is "administrator" user lockout.

Users of this tool need to be aware that using repository layer, access is not validated using authorisations, model component is not used during processing and changes are not audited. It allows you to obtain or modify data in an otherwise unusable midPoint. Ninja also plan to support some other operations like listing keys from keystore, enforcing objects reencryption with new key, DB schema validation, password reset or user unlock.

Ninja can be started using standard java -jar or via bundled shell/bat script. Bundled shell and bat scripts are located in the midPoint installation directory, e.g. <midpoint>/bin/ninja.sh. Bundled scripts will automatically fill in path to for midPoint home option -m using MIDPOINT_HOME environment variable, defaulting to the <midpoint>/var.

Connecting to midPoint

Connection options:

  • using midpoint.home

  • using REST service (not yet implemented)

midpoint.home connection

This type of connection initialises repository layer of midPoint and operates on top of the database as a new midPoint node. Mostly only one option is needed and that’s specification of midpoint.home directory using -m option. This is not sufficient if config.xml uses datasource definition. Ninja then doesn’t know how to connect to DB. For this case one have to use options to specify URL (-U), username (-u) and password (-p or -P). These options can also be used to override JDBC URL, username and password specified in config.xml.

With bash, you can provide MIDPOINT_HOME for the ninja.sh command with this shortcut:

MIDPOINT_HOME=/var/other-mp-home <midpoint>/bin/ninja.sh [general options] [command] [command options]

If unbundled JDBC driver is used, it will not be picked up automatically from the midPoint home directory and you must point Ninja to it explicitly. Example when using java -jar:

java -Dloader.path=<jdbc_driver_jar_path> -jar ninja.jar [general options] [command] [command options]

If the bundled scripts are used, you can specify the path to the driver with -j option, for example:

<midpoint>/bin/ninja.sh -j <JDBC_DRIVER_JAR> [general options] [command] [command options]

Supported operations

Ninja currently supports these operations:

  • count

  • delete

  • download-distribution (midPoint 4.4.6, 4.7.2, 4.8 or later)

  • export

  • export-audit (midPoint 4.4.1 or later)

  • export-mining (midPoint 4.4.4 or later)

  • help (midPoint 4.8 or later)

  • import

  • import-audit (midPoint 4.4.1 or later)

  • info

  • keys

  • pre-upgrade-check (midPoint 4.4.6, 4.7.2, 4.8 or later)

  • run-sql (midPoint 4.4.6, 4.7.2, 4.8 or later)

  • trace (midPoint 4.4.6, 4.7.2, 4.8 or later)

  • upgrade-distribution (midPoint 4.4.6, 4.7.2, 4.8 or later)

  • upgrade-installation (midPoint 4.4.6, 4.7.2, 4.8 or later)

  • upgrade-objects (midPoint 4.4.6, 4.7.2, 4.8 or later)

  • verify (midPoint 3.9 or later)

Import

Import command allow users to import objects from SYSOUT or file (either plain XML or compressed ZIP, which may contain multiple files). One can define oid, multiple types or filter to filter imported objects. Example of import from XML file using 4 threads with raw and overwrite option using explicit midpoint.home folder path:

Example
java -jar ninja.jar -m ~/Desktop/midpoint/var import -O -i midpoint/samples/objects/user-ceresnickova.xml -l 4 -r

Export

Export command allows users to export objects to SYSOUT or file (either plain XML or compressed ZIP). One can define oid, multiple types or filter to filter exported objects. Export objects to compressed zip file using 4 threads:

Example
java -jar ninja.jar -m ~/Desktop/midpoint/var export -O export.zip -z -l 4

Use -O switch if you wish to overwrite an existing output file.

Verify

MidPoint 3.9 and later

This feature is available only in midPoint 3.9 and later.

Command that verifies objects in midPoint repository. It displays warnings about objects to SYSOUT or file. Similarly to export, one can define oid, multiple types or filter to filter verified objects. Simple usage to verify all objects in the repository for all warnings:

Example: verify all objects in repository for all warnings
ninja.sh verify

There is a -w switch that can be used to select specific warnings to display. Currently, it only supports values deprecated and plannedRemoval. Following command will show warnings about planned removal of items used by all objects in the repository:

Example: verify all objects in repository for planned removal of items
ninja.sh verify --verification-category planned_removal

Keys

List keys in with aliases from keystore located in midpoint.home.

Example
java -jar ninja.jar -m /opt/midpoint-home keys -K

Audit export/import

Audit export/import uses the same format as objects, even utilizing <objects> element, but audit records are containers, not objects. These files are only useful for Ninja import/export, do not try to use them in Studio or otherwise.
Example: audit export to a zipped file, overwriting any existing file, using an Axiom filter
ninja.sh exportAudit -o audit-export.xml.zip -z -O -f '%timestamp < "2021-12-01"'

Now you can move the exported file to a new environment and run the import:

Example: audit import
ninja.sh importAudit -i audit-export.xml.zip -z

Check also the migration guide for the Native audit.

Role mining export/import

Role mining export is an anonymized export of relationships between roles, users, and organizations. The goal of the export is to obtain a map of relations between objects, which could be helpful in the development of new features such as "Role mining". For detailed information regarding role mining export, please refer to the Anonymous Export of Role Mining Data.
Example: role mining export to a zipped file, overwriting any existing file. SEQUENTIAL name mode and ADVANCED security by default. For exporting in JSON format, rename the exported file extension to -o role-mining-export.json.
ninja.sh export-mining -o role-mining-export.xml.zip -z -O
java -jar ninja.jar -m <midpoint.home> export-mining -o role-mining-export.xml.zip -z -O
Example: role mining export using ENCRYPTED name mode and STANDARD security.
ninja.sh export-mining -o role-mining-export.xml.zip -z -O -nm ENCRYPTED -s STANDARD
Example: role mining export using an Axiom filter for roles.
ninja.sh export-mining -o role-mining-export.xml -fr '% name != "Superuser"'
Example: role mining export using ORIGINAL name mode, identifiers Application and Business role prefixes/suffixes and specific archetypes oids.
ninja.sh export-mining -o role-mining-export.xml  -nm ORIGINAL -arp "APP_ROLE_, AR-" -ars "_Apr" -brp "BUS_ROLE_" -brs "_BR" --business-role-archetype-oid "e9c4654e-c146-4b5f-8336-2065c65060df" --application-role-archetype-oid "52b8361a-c955-4132-97a4-77ff3820beeb"
Example: role mining export with disable organization structure export.
ninja.sh export-mining -o role-mining-export.xml --disable-org
Example: role mining import.
java -jar ninja.jar -m <midpoint.home> import -O -i <path_to_imported_file> -l 4 -r

Other scenarios

Administrator unlock

Ninja does not provide unlock or password reset operations explicitly. Normally administrator can do this via GUI. It may happen that administrator locks out itself from the system, in which case we recommend to reinitialize the object:

Administrator object refresh from the initial object
ninja.sh import --input <midpoint>/doc/config/initial-objects/050-user-administrator.xml --overwrite

Now, this is rather a drastic measure - let’s modify the existing user instead. First we need to export it.

If you want to use Ninja for midPoint running with H2 database (which is sensible only for testing), the application must be stopped first. Other databases have dedicated servers and Ninja can be used while midPoint is running.
Exporting administrator user by OID
ninja.sh export --oid 00000000-0000-0000-0000-000000000002 --output admin.xml

Alternatively, you can use type option and filter to specify name (don’t forget the --overwrite option, if the file already exists):

Exporting user by name
ninja.sh export --type user --filter '% name = "administrator"' --output admin.xml

Now is the time to carefully edit the object XML. If a single object is in the XML (which is our case), you can remove the <objects> wrapper element. This may actually help with code completion if midPoint Studio is used.

Typical fixes to shape up the uncooperative user object are:

  • Remove administrativeStatus and effectiveStatus elements from the top-level activation element (be sure not to modify activation under assignments inadvertently). This should fix any problems with disabled user.

  • If you forgot the password, change the whole content of the credentials/password/value element to this snippet (must be inside the value element):

    <t:clearValue>asdf1234</t:clearValue>

After the fixes, it’s time to push the object back into the repository. Import process can handle both plain object or objects inside <object> container. Use the following command:

Importing the fixed user object
ninja.sh import --input admin.xml --overwrite

Now you should enjoy easy administrator login. Of course, this works for any user object, but using the GUI with working administrator is the way to go.

If you changed the password in this way, don’t forget to change it immediately to something more solid. This will also properly encrypt it in the object representation, as expected.

Full help

Full help
$ java -jar target/ninja.jar help
Usage: java [-Dloader.path=<jdbc_driver_jar_path>] -jar ninja.jar [options] [command] [command options]
  Options:
    -B, --batch-mode
      Run in non-interactive (batch) mode (disables output color).
      Default: false
    -c, --charset
      Charset used for input/output
      Default: UTF-8
    -h, --help
      Print this help
    -m, --midpoint-home
      Path to MidPoint home folder. If relative path is specified, it will be translated to absolute path.
    -p, --password
      Password for JDBC connection
    -P, --password-ask
      Ninja will ask for JDBC connection password
    --psn-class-name
      PolyString normalizer class name
    --psn-lowercase
      PolyString normalizer lowercase
    --psn-nfkd
      PolyString normalizer NFKD. Unicode Normalization Form Compatibility Decomposition (NFKD). This process
      expands composed characters (e.g. diacritic marks, ligatures, roman numerals). Character components are
      then processed individually. E.g. ascii7 normalizer will then remove just the diacritic mark, but leave
      the base letter. Therefore, for a-acute character with NFKD processing the output will be plain "a"
      character. Without NFKD processing the whole character will be removed.
    --psn-trim
      PolyString normalizer trim. Removes whitespace from beginning and end of the string. Trimming is done
      before any other processing.
    --psn-trim-whitespace
      PolyString normalizer trim whitespace
    -s, --silent
      No output at all
      Default: false
    -U, --url
      JDBC URL to database. If '-m' option is used URL will be used to connect to JDBC database.
    -u, --username
      Username for JDBC connection
    -v, --verbose
      Verbose output
      Default: false
    -V, --version
      Version and build description

  Commands:

    import                  Imports objects into MidPoint
    export                  Exports objects from midPoint
    delete                  Delete objects from MidPoint
    count                   Count objects in midPoint repository
    verify                  Verify objects in midPoint repository
    keys                    List keys from keystore
    info                    Shows information about the repository for the provided midPoint home
    import-audit            Imports audit records into MidPoint
    export-audit            Exports audit records from MidPoint
    export-mining           Export of anonymized structure of relationships between roles, users and
            organizations objects.
    trace                   Edit trace file
    download-distribution   Download target midPoint distribution (latest LTS).
    run-sql                 Run arbitrary SQL scripts
    upgrade-installation    Upgrade installation
    upgrade-distribution    Upgrade midPoint distribution and related DB schema
    upgrade-objects         Upgrade objects
    pre-upgrade-check       Pre-upgrade check
    help                    Print this help, or prints help for specific command.
    initial-objects         Initial objects
Was this page helpful?
YES NO
Thanks for your feedback