Integration Tests

Almost all integration tests are creating and initializing embedded midPoint instance. That means that the tests are creating instances of all the midPoint components that they can. In fact this initialization is mostly done by the spring context. You may notice spring configuration files in the src/test/resources directories. These are used to create the embedded midPoint instance.

The tests must populate this database with a configuration. This creates the test configuration and environment. Many test families already have superclasses that create the part of the configuration that is common for that particular family.

There are several test superclasses that contains the code that initializes the embedded midPoint. The superclasses also contain a lot of utility methods to manipulate repository objects, lot of assertion methods and so on. The superclasses are:

Your test code can directly subclass any of these abstract superclasses. The tests come with empty repository, no resources and just the basic connectors.

There are also abstract superclasses that are pre-configured for a specific setup, e.g. many tests in model-intest component use AbstractInitializedModelIntegrationTest that contains a code that pre-populate the repository with a common set of objects.

The integration tests are initialized using the initSystem() method. This method can be used to insert initial objects into repository, initialized the resources, and so on.

Tests methods are executed sequentially in an alphabetical order. It is a convention to start the name of a test method with testXXX where XXX is a decimal number. The test execution is ordered by number. The number should be followed with a short name that describes what the test is trying to do. E.g.:

The methods in a test class should form a story. Contrary to the frequently recommended practice the tests methods may depend on one another. E.g. one method may assign a role to a user which creates an account, following method may try to retrieve the account, next method may try to modify the account and the last method may try to unassign the role which deletes the account. This makes the test code easy to write and reasonably easy to maintain. The drawback is that if the first tests fails other tests will also fail. But we can live with that.

There is a convention to use test method test000Sanity to check that the test environment was initialized properly. This tests is like a pre-condition for the entire test class.

Every test is testing some kind of scenario or story. If the story is not obvious from the test code at the first sight then please describe the scenario in one or two sentences. Like this:

Please also specify Jira-style (MID-xxxx) issue identifier if it is applicable. This may be an issue that refers to the bug that you are trying to reproduce in the test. Or it may be ID of a feature that you are trying to implement.

Good practice is to divide each test method to three parts:

It is recommended to visibly mark the tests parts with comments // GIVEN, // WHEN and // THEN. Especially the WHEN mark is important. Code is the best documentation. Marking the core part of the test allows the developer to figure out what the test does at a first sight. Test maintenance is a major task. Therefore please save the time of your colleagues (and yourself). There are also appropriate methods to dump label for a started test and when/then section. These methods dump markers to logfiles and test output. Therefore it is easier to locate corresponding parts of test output after it is executed.

The test code should look like this:

The test should be completely self-sufficient when it comes to checking the tests results. E.g. use assert methods to check that the test did what it should do. But, the tests need to be maintained and it is often good to display entire objects to help diagnose the test problems. There is a variety of display() methods especially for this purpose. All the display methods write the data both to test log and to standard output.

The display() method can be used to display almost any value that is used in midPoint. The method will make sure that the value is displayed in human-readable form. Also a title can be specified. E.g.:

display("Repository shadow", shadow)

will display the full dump of the PrismObject<ShadowType> object that is in the shadow variable. This a good way how to display diagnostic information about the test progress.

The displayTestTitle(), displayWhen() and displayThen() will mark tests and test parts. It is very good when examining test logfile, because simple test search can be used to skip to the start of the test or start of test part.

Tests often use files to initialize the repository, store queries, deltas and so on. The files should be placed in src/test/resources subdirectories. However placing all the files in one directory will create a chaos. Therefore we use subdirectories:

Tests are using generic H2 repository by default. This repository does not support many midpoint features. Therefore, native PostgreSQL repository is needed for relevant test run.

Native PostgreSQL repository can be enabled by enabling sqale maven profile.

PostgreSQL databse has to be available on localhost. Default database name is midtest.

Each test class will refresh spring context, reinitialize repository instance and clear repository database. By default, repository starts as completely empty. There is even no system configuration or an administrator user. These should be explicitly added in initSystem() method if they are needed.

Make sure there is @DirtiesContext(classMode = ClassMode.AFTER_CLASS) annotation at the class level for each integration test. This makes sure that spring context is re-initialized after the tests and that the test will not leave dirty repository for the following tests.

Model integration tests contain an initialized instance of provisioning components. This includes the usual set of connectors (LDAP, DB, CSV). There is also a special connector and an associated resource especially designed for integration tests: Dummy Resource.

The Dummy Resource simulates a real resource by using in-memory maps and lists. Therefore it is very lightweight. It supports accounts, groups and privilege object classes. It has extensible schema. It can behave in very tolerant and also in a very strict modes (e.g. being case in-sensitive, tolerate duplicates or not tolerate anything at all). This can all be configured by using connector configuration properties. It can also simulate network (and other errors). Therefore it is ideal for the tests. Have a look at TestDummy (provisioning-impl) or any test in model-intest component to see how to the dummy connector and resource are used.

There is also option to use embedded LDAP server (OpenDJ) and database (Derby) in tests. Have a look at TestOpenDJ and TestDBTable in provisioning-impl.

All the tests log to the same log file:

target/test.log

The logging levels are usually controlled by the file:

src/test/resources/logback-test.xml

This file controls logging in majority of integration tests. However, there are few exception that need to initialize the system in slightly a different way. In that case the logging configuration from the SystemConfiguration object in the repository is used.

Test output (stdout and stderr) are stored in files:

target/surefire-reports/-output.txt*

The test output usually contains just the things that are printed by display() methods. Therefore the test output is usually much faster way to diagnose test problems as the objects are seen almost at the first sight. However the test output does not contain the details. Therefore the usual procedure is to look at the test output to get overall idea what is going on and then have a look at the log files. If the display() methods are used properly that the log files also contains all the test output, test markers, test part markers and so on. So simply searching for a test name in the log file will get you at the right place in the logfile.

In the code Logger is available via protected instance variable logger in any subclass of AbstractSpringTest or AbstractUnitTest. This logger always reflects the real test class name. There is no need to create variables in concrete classes, unless you need to log in static context - which can (and should) be avoided in most situations.

Though shalt not disable the tests! Do not disable the tests unless there is very very very good reason for it. Perhaps the only good reason for disabling a test is this: you have created a new test and that test have found a nasty bug that you cannot fix right now. The bug is not critical and you do not want the whole build to fail. Then do this:

Never ever commit a disable test without the reference to the issue.