Java Connector Server

Last modified 09 Aug 2024 12:59 +02:00
Table of Contents

Java based ConnId server for remote connector execution

Functionalitystable
Development statusactive (actively developed and maintained)
Support statussupportable
OriginconnId
Support provided byEvolveum
Source codehttps://github.com/Evolveum/ConnId

Java Remote Connector Server is using the same kind of connectors that midPoint itself is using. The Java Remote Connector Server is used in situations where a connector needs a local access to some resource to be able to work with it. It is usually used for connectors that require local access to files such as CSVFile Connector (legacy). This avoids the need to copy the file using FTP or a similar mechanism which is difficult to do right and it is quite error-prone (e.g. problems with partially downloaded files, error handling, atomicity, etc.) Java Remote Connector Server is also used in situations that require firewall traversal or securing insecure communication protocol.

Connector Server support is related to Connector Server submodules present in the connId project since version 1.5.1.0. The prerequisite for the correct functionality of the connector server is a midPoint instance running with this or a more recent version of connId. I.e. since midPoint 4.6+

Requirements

  • Java SE 6 or later

  • (from connId 1.5.1.0) Java SE 11 or later

Download

Version Download Sources Note

1.5.1.10

ZIP

https://github.com/Evolveum/ConnId/tree/connid-1.5.1.10

require midPoint 4.6+ or 4.4.4 (not working with 4.5)

1.5.0.18

ZIP

https://github.com/Evolveum/ConnId/tree/connid-1.5.0.18

1.4.2.35

ZIP

https://github.com/Evolveum/ConnId/tree/connid-1.4.2.35

1.4.0.49

ZIP

https://github.com/Evolveum/ConnId/tree/connid-1.4.0.49

1.1.1.0

ZIP

https://github.com/ForgeRock/openicf-java-framework-community-edition/tree/fr/1.1.1.0

obsolete version, probably won’t work with current midPoint

You may also download from the OpenICF download page.

In case you are working with containers ( e.g. docker, kubernetes…​ ) there is available dockerfile to build the image on gitHub. The dockerfile is tested for versions 1.5.0.18 (at this moment default ARG values) and for the 1.5.1.10 (at this moment not compatible with the latest released version of midPoint).

Installation (until connId 1.5.1.0)

  1. Download and unzip the binary distribution (or clone git repo with sources and build your own with mvn clean install command)

  2. In the installation folder (that contains bin, conf, and lib directories) create a directory for connector bundles, named bundles. In the following text, we assume /opt/connid-connector-server directory for Linux.

  3. Copy connectors you need into bundles directory (e.g. connector-csv-2.4.jar for CSV connector)

  4. Set the secret key by invoking the command:

    1. (on Windows): bin\ConnectorServer.bat /setkey <your secret key here>

    2. (on Linux): java -cp "lib/framework/connector-framework.jar:lib/framework/connector-framework-internal.jar:lib/framework/groovy-all.jar" org.identityconnectors.framework.server.Main -setKey -key <your secret key here> -properties conf/ConnectorServer.properties

  5. Fix the logging configuration:

    1. replace the line “connectorserver.loggerClass=org.identityconnectors.common.logging.slf4j.SLF4JLog_" in conf/ConnectorServer.properties file with "_connectorserver.loggerClass=org.identityconnectors.common.logging.impl.JDKLogger”

    2. add "-Djava.util.logging.config.file=conf/logging.properties" to your startup parameters to actually use logging

    3. update the conf/logging.properties to log to file in logs directory:[source]

handlers=java.util.logging.FileHandler
##handlers=java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
java.util.logging.FileHandler.pattern = logs/connectorserver%u.log
java.util.logging.FileHandler.limit = 102400
java.util.logging.FileHandler.count = 1
java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter
java.util.logging.FileHandler.append = true
.level=INFO

  1. Run the connector server e.g. by invoking the command:

    1. (on Windows): bin\ConnectorServer.bat /run

    2. (on Linux): java -cp "lib/framework/connector-framework.jar:lib/framework/connector-framework-internal.jar:lib/framework/groovy-all.jar" org.identityconnectors.framework.server.Main -run -properties conf/ConnectorServer.properties

Connector Server will run on foreground/console. CTRL+C will stop it.

Installation

Since 1.5.1.0 (connId)
This functionality is available since version 1.5.1.0 of connId.
This version is compatible with midPoint 4.6+

  1. Download and unzip the binary distribution (or clone git repo with sources and build your own with mvn clean install command)

  2. After the project is built you should find a zip file under /ConnId/java/connector-server-zip/target named connid-connector-server-java-CONNID-VERSION.zip

  3. Unzip this file in your preferred directory

  4. The name of the resulting sub-directory is connid-connector-server/ and it contains the following child directories:

    1. bin, which contains the connector server startup scripts

    2. bundles, a directory to which you copy the desired "connector.jar" files (connectors)

    3. conf, configuration properties are stored here

    4. lib, contains the logback configuration files and also a subdirectory with java libs which are leveraged by the connector server

  5. To start the server itself you need to invoke one of the startup scripts in the "bin/" subdirectory

    1. For Linux environments execute the .sh script with following command parameters ./bin/ConnectorServer.sh -run -properties conf/connectorserver.properties, also add -setKey -key <server password here> to change server password to other than default

    2. For Windows environments execute the .bat script with run command bin\ConnectorServer.bat /run, also add /setKey <server password here> to change server password to other than default

  6. After the script execution a "/logs" directory is created with .log generated based on the logback.xm configuration, the defaults are:

    1. Logging of the connector server related libraries is dumped to "ConnectorServer.log"

    2. Logging of the connector instances is dumped to "Connector.log"

Connector Server will run on foreground/console. CTRL+C will stop it.

Server Properties

Since 1.5.1.0 (connId)
This functionality is available since version 1.5.1.0 of connId.

The connector server can be configured via the connectorserver.properties file which resides in the connid-connector-server/conf sub-directory. This contains the following parameters with defaults (each with a prefix of 'connectorserver.'):

  1. port [default '8759'], the port on which to execute

  2. bundleDir [default 'bundles'], path to directory where to find the connector bundles

  3. libDir [default 'lib'], path to directory where to find the libraries needed at runtime

  4. usessl [default 'false'], true if connector server should use SSL, please see "Configuring SSL"

  5. key [default 'lmA6bMfENJGlIDbfrVtklXFK32s\=', e.g. 'changeit'], secure hash of the gateway key

    1. can be changed by the option '-setKey -key' (Linux) or '/setKey' (Windows) added to the script execution

  6. loggerClass [default 'org.identityconnectors.common.logging.impl.JDKLogger'], logger used by the connector server, there are multiple options:

    1. org.identityconnectors.common.logging.impl.JDKLogger, [default] can be configured via logback.xml (jul-slf4j bridge)

    2. org.identityconnectors.common.logging.impl.noOpLogger, no logging

    3. org.identityconnectors.common.logging.StdOutLogger, logging to standard output

    4. org.identityconnectors.common.logging.slf4j.SLF4JLog, can be configured via logback.xml

  7. ifaddress [optional and by default not used, e.g. 'localhost'], specific address to bind to

Logback configuration

Since 1.5.1.0 (connId)
This functionality is available since version 1.5.1.0 of connId.

Using either the JDKLogger [default] or SLF4JLog you are capable of configuring the logging properties via a logback.xml configuration file. The file is present in the connid-connector-server/lib subdirectory. This is a part of the default configuration present in the execution scripts ConnectorServer.sh a ConnectorServer.bat. In both cases it’s a result of specifying the -Dlogback.configurationFile=lib/logback.xml java property. If this is removed by default the logback.groovy configuration file is used as default [as stated in logback documentation].

I will describe the 'logback.xml' as this is the current default used during startup.

There are three main appenders, "SERVER-FILE", "CONNECTOR-FILE" and "STDOUT". The "STDOUT" appender is used as the root appender, currently all unspecified packages dump messages with the "debug" level to this appender. "SERVER-FILE" contains log messages related to the server libraries itself. This is a file appender for the 'connid-connector-server/logs/ConnectorServer.log' file. The verbosity of most of the loggers in this appender are governed by the 'SERVER_LEVEL' logback property set by defatul to the "INFO" level. "CONNECTOR-FILE" is the appender used to dump the messages originating from the actions of identity connector bundles. This is dumped to the file 'connid-connector-server/logs/Connector.log'. this case you might need to add also a logger to the 'polygon' project packages to log connectors base on the midPoint polygon bundles. Following is an example to set the "polygon" packages to the "TRACE" level. In this case all connectors based on the packages will bump trace level logs into the log file.

 <logger name="com.evolveum.polygon" level="TRACE" additivity="false">
    <appender-ref ref="CONNECTOR-FILE"/>
 </logger>

Using connectors which require SSL

In this case you need to set up a keystore file where you should store the needed ssl certificates. This does not require the change of the 'usessl' configuration property to true. What you additionally need to set up is the specification of the keystore file which should be used and it’s properties in the execution scripts. See 'Passing Keystore Parameters to Connector Server'.

Best results here were by using the keystore format PKCS12, the type JCEKS seemed to cause some issues.

Automatic Server Startup

Systemd

Create user/group for running the service (e.g. connid-server, connid-server). The home directory of the connector server should be located in the directory "/opt/connid-connector-server" in case of this example. Also set the "/opt/connid-connector-server/bin/ConnectorServer.sh" script to be executable. This user must have access to the connector server files.

Create systemd service file /etc/systemd/system/java-connector-server.service (as root) - inspiration from http://stackoverflow.com/questions/21503883/spring-boot-application-as-a-service/22121547#22121547:

until connId 1.5.1.0

[Unit]
Description=Java Connector Server Service
[Service]
User=connid-server
WorkingDirectory=/opt/connid-connector-server
ExecStart=/usr/bin/java -Xmx256m -cp "lib/framework/connector-framework.jar:lib/framework/connector-framework-internal.jar:lib/framework/groovy-all.jar" org.identityconnectors.framework.server.Main  -run -properties conf/ConnectorServer.properties
SuccessExitStatus=143
[Install]
WantedBy=multi-user.target

from connId 1.5.1.0

[Unit]
Description=Java Connector Server Service
[Service]
User=connid-server
WorkingDirectory=/opt/connid-connector-server/
ExecStart=/opt/connid-connector-server/./bin/ConnectorServer.sh -run -properties /opt/connid-connector-server/conf/connectorserver.properties
SuccessExitStatus=143
[Install]
WantedBy=multi-user.target

Issue the following commands (as root):

systemctl daemon-reload
systemctl enable java-connector-server

You can start/stop the service using:

systemctl start java-connector-server
systemctl stop java-connector-server

SysV Init

Create start script to be run by startup script /opt/connid-connector-server/start:

#!/bin/bash
MAIN_DIR=/opt/connid-connector-server
cd $MAIN_DIR
exec java -Djava.util.logging.config.file=conf/logging.properties -cp "lib/framework/connector-framework.jar:lib/framework/connector-framework-internal.jar:lib/framework/groovy-all.jar" org.identityconnectors.framework.server.Main -run -properties conf/ConnectorServer.properties

Set file permissions:

chmod 755 /opt/connid-connector-server/start

Create startup script /etc/init.d/connid-connector-server - inspiration from: https://orrsella.com/2014/11/06/initd-and-start-scripts-for-scala-java-server-apps/

#!/bin/bash
START_SCRIPT=/opt/connid-connector-server/start
PID_FILE=/var/run/connid-connector-server.pid
DAEMON=$START_SCRIPT
start() {
  PID=`$DAEMON $ARGS > /dev/null 2>&1 & echo $!`
}
case "$1" in
start)
    if [ -f $PID_FILE ]; then
        PID=`cat $PID_FILE`
        if [ -z "`ps axf | grep -w ${PID} | grep -v grep`" ]; then
            start
        else
            echo "Already running [$PID]"
            exit 0
        fi
    else
        start
    fi
    if [ -z $PID ]; then
        echo "Failed starting"
        exit 3
    else
        echo $PID > $PID_FILE
        echo "Started [$PID]"
        exit 0
    fi
;;
status)
    if [ -f $PID_FILE ]; then
        PID=`cat $PID_FILE`
        if [ -z "`ps axf | grep -w ${PID} | grep -v grep`" ]; then
            echo "Not running (process dead but pidfile exists)"
            exit 1
        else
            echo "Running [$PID]"
            exit 0
        fi
    else
        echo "Not running"
        exit 3
    fi
;;
stop)
    if [ -f $PID_FILE ]; then
        PID=`cat $PID_FILE`
        if [ -z "`ps axf | grep -w ${PID} | grep -v grep`" ]; then
            echo "Not running (process dead but pidfile exists)"
            exit 1
        else
            PID=`cat $PID_FILE`
            kill -HUP $PID
            echo "Stopped [$PID]"
            rm -f $PID_FILE
            exit 0
        fi
    else
        echo "Not running (pid not found)"
        exit 3
    fi
;;
restart)
    $0 stop
    $0 start
;;
*)
    echo "Usage: $0 {status|start|stop|restart}"
    exit 1
esac

Set file permissions:

chmod 755 /etc/init.d/connid-connector-server

Start the service:

/etc/init.d/connid-connector-server start

Set the service to autostart (using your distribution command; here Red Hat-based distributions "chkconfig" is used:

chkconfig connid-connector-server on

You may need to use different command and edit the script to use dependencies or service startup ordering.

Original instructions for OpenICF Connector Server: http://openicf.forgerock.org/connector-framework-internal/connector_server.html

Configuring SSL

The Connector Server is a SSL server. Therefore is needs a keypair (private key + certificate). Java connector server expects the keypair to be present in a keystore. It is using standard Java JCE keystore for this purpose. The keystore does not exist at the time of the initial installation. It needs to be created and populated with a keypair.

Creating and Populating a Keystore

The keypair is usually distributed in a PKCS#12 format (a file with p12 or pfx extension). This format needs to be converted in Java JCE keystore. There is keytool utility that is part of Java platform that can be used for conversion:

Converting PKCS#12 key and certificate to java keystore
keytool -importkeystore -srckeystore mykeycert.p12 -srcstoretype pkcs12 -destkeystore keystore.jks -deststoretype JKS

The command above creates a keystore.jks file which is the actual Java JCE keystore. The keytool command will ask for two passwords:

  • A password on the PCKS#12 files as these files are usually protected by password (because they contain a private key)

  • A password for a newly created keystore. Make sure you remember this.

But there is a catch. The Java JCE keystore as a whole is protected by a password. But also each individual key is protected by a password. These passwords are usually the same and that is exactly what the connector server expects. However when the keystore is converted from PCKS#12 the keystore password is set to the supplied password but the key password remains the same as was the password on PCKS#12 file. If these passwords were not the same then the key password needs to be changed in one extra step:

Changing a key password
keytool -keystore keystore.jks -storepass changeit -keypasswd -alias mykey

See Keystore Configuration page for some more tips and tricks dealing with keystore. But please note that this page deals with midPoint keystore which is slightly different than Connector server keystore.

Passing Keystore Parameters to Connector Server

The connector server is a Java application that looks for a default keystore. The location, type and password of the default keystore needs to be passed to the connector server in a form of Java options:

java ... -Djavax.net.ssl.keyStore=keystore.jks -Djavax.net.ssl.keyStorePassword=changeit -Djavax.net.ssl.keyStoreType=JKS ...

Add these options to the script that is starting connector server.

Enabling Connector Server SSL

Change the connectorserver.usessl option to true in the connectorserver.properties configuration file.

You can start the server now. Please do not forget to configure the midPoint side as well.

Troubleshooting

Error "Cannot recover key": Make sure that the key password in the keystore is the same as the keystore password.

Using remote connectors

To integrate remote connectors to your midPoint instance we use a special type of object called Connector Host. This represents information about the connector server, specifically it represents a configuration which enables us to communicate with the remote server.

When the connector host object is configured we are capable of executing via this object the action of connector discovery. This causes midPoint to request the remote connector server instance for information about possible connector bundles available at the remote location. With this information midPoint creates the connector object representation in its repository, containing oid’s same as other objects in midPoint. You are then capable of using those or other attributes to specify the needed connector in your resource configuration.

The created connector object will have the name of the connector host as a part of its own name.

Some an example and more information about the connector host object can be found here.

See Also

Was this page helpful?
YES NO
Thanks for your feedback