Connecting Syslog-ng and WSO2 EI

This series of topics will be about connecting the Syslog server and WSO2 EI and establishing mutual authentication using TLS. Following are the topics that we will be focusing on to,

Topic 1 : Building ESB artifacts

Creating an ESB project

Creating the sample message template

Create a sample ESB project and create the content that you want to send to the syslog server. In this article, I have used a sequence template to create the sample message. Following is the sample message that will be sent to the syslog server.

Hint : Use a payload factory mediator to build the sample message that you want to send.

After creating the message add the following configuration to invoke the class mediator,

<class name="com.poc.logging.SyslogMediator"/>

Create the sample proxy service

Creating a sample proxy service to invoke the message template

<?xml version="1.0" encoding="UTF-8"?>
<proxy name="hl7testproxy" startOnLoad="true" transports="https http hl7" xmlns="http://ws.apache.org/ns/synapse">
    <target>
        <inSequence>
            <log level="custom">
                <property name="STATUS" value="****insequence started*****"/>
            </log>
            <sequence key="ITI-AuditMessage_Template"/>
            <log level="full"/>
            <log level="custom">
                <property name="STATUS" value="****insequence ended"/>
            </log>
        </inSequence>
        <outSequence>
            <log level="custom">
                <property name="STATUS" value="****outsequence started*****"/>
            </log>
            <log level="full"/>
            <send/>
        </outSequence>
        <faultSequence/>
    </target>
</proxy>

Build and deploy the above in the WSO2 EI

Creating a Sample Class mediator

The sample class mediator code is as follows,

package com.poc.logging;
import org.apache.synapse.MessageContext; 
import org.apache.synapse.mediators.AbstractMediator;
public class SyslogMediator extends AbstractMediator { 
    
    public SyslogMediator() {
    }
    public boolean mediate(MessageContext mc) {
        try {
            log.info(mc.getProperty("syslogMessage"));
        } catch (Exception e) {
            log.error("ERROR :", e);
        }
        return true;
    }
}

Build and deploy the class mediator to the /{carbon_home}/lib location and restart the WSO2 EI. Makesure that the plugin folder is also updated with the mediator.

Configuring the log4j2 file

Create a custom appender and add the below configurations in order to print the mediator logs in the carbon log file.

1. Add the custom appender to the list of all appenders as below

2. In the loggers section add the following configuration to print the logs of the Mediator.

3. Configure the logger as below

logger.com-poc-logging-SyslogMediator.name=com.poc.logging.SyslogMediator
logger.com-poc-logging-SyslogMediator.level=INFO
logger.com-poc-logging-SyslogMediator.appenderRef.WSO2AUDIT_LOGFILE.ref = WSO2AUDIT_LOGFILE

Creating the custom appender

Create the custom appender configuration as below. Enable port 6514 for WSo2 EI communication over TLS. Following is the full code for log4j2 configuration file.

#custom syslog appender
appender.WSO2AUDIT_LOGFILE.type = Syslog
appender.WSO2AUDIT_LOGFILE.name = WSO2AUDIT_LOGFILE
appender.WSO2AUDIT_LOGFILE.host = 127.0.0.1
appender.WSO2AUDIT_LOGFILE.port = 6514
appender.WSO2AUDIT_LOGFILE.layout.type = PatternLayout
appender.WSO2AUDIT_LOGFILE.layout.pattern = [%d] [%tenantId] %5p {%c} - %m%ex%n
appender.WSO2AUDIT_LOGFILE.filter.threshold.type = ThresholdFilter
appender.WSO2AUDIT_LOGFILE.filter.threshold.level = DEBUG

Following is the configuration to enable mutual authentication for the EI and the syslog server.

appender.WSO2AUDIT_LOGFILE.SslConfiguration.type = Ssl
appender.WSO2AUDIT_LOGFILE.SslConfiguration.KeyStore.type = KeyStore
appender.WSO2AUDIT_LOGFILE.SslConfiguration.KeyStore.location = /home/wso2/WSO2/keystore-backup/client.jks
appender.WSO2AUDIT_LOGFILE.SslConfiguration.KeyStore.password = wso2keystore
appender.WSO2AUDIT_LOGFILE.SslConfiguration.TrustStore.type = TrustStore
appender.WSO2AUDIT_LOGFILE.SslConfiguration.TrustStore.location = /home/wso2/WSO2/keystore-backup/service.jks
appender.WSO2AUDIT_LOGFILE.SslConfiguration.TrustStore.password = wso2keystore

Following is the entire log4j appender configuration for WSo2 EI mutual authentication over TLS.

#custom syslog appender
appender.WSO2AUDIT_LOGFILE.type = Syslog
appender.WSO2AUDIT_LOGFILE.name = WSO2AUDIT_LOGFILE
appender.WSO2AUDIT_LOGFILE.host = 127.0.0.1
appender.WSO2AUDIT_LOGFILE.port = 6514
appender.WSO2AUDIT_LOGFILE.layout.type = PatternLayout
appender.WSO2AUDIT_LOGFILE.layout.pattern = [%d] [%tenantId] %5p {%c} - %m%ex%n
appender.WSO2AUDIT_LOGFILE.filter.threshold.type = ThresholdFilter
appender.WSO2AUDIT_LOGFILE.filter.threshold.level = DEBUG
appender.WSO2AUDIT_LOGFILE.SslConfiguration.type = Ssl
appender.WSO2AUDIT_LOGFILE.SslConfiguration.KeyStore.type = KeyStore
appender.WSO2AUDIT_LOGFILE.SslConfiguration.KeyStore.location = /home/wso2/WSO2/keystore-backup/client.jks
appender.WSO2AUDIT_LOGFILE.SslConfiguration.KeyStore.password = wso2keystore
appender.WSO2AUDIT_LOGFILE.SslConfiguration.TrustStore.type = TrustStore
appender.WSO2AUDIT_LOGFILE.SslConfiguration.TrustStore.location = /home/wso2/WSO2/keystore-backup/service.jks
appender.WSO2AUDIT_LOGFILE.SslConfiguration.TrustStore.password = wso2keystore

Topic 2 : Deploying and Managing syslog-ng server

Following is the command to deploy the syslog server in a linux environment. For this article, I have tested this implementation in a Ubuntu(18.04) environment.

Install syslog-ng and any of its subpackages:

sudo apt-get install syslog-ng-core

Reference : https://www.syslog-ng.com/community/b/blog/posts/installing-the-latest-syslog-ng-on-ubuntu-and-other-deb-distributions

By the above the syslog server will be installed in the /etc/syslog-ng location. Add the following configurations to the syslog-ng.conf in the product home.

source s_network {
        default-network-drivers(
                # NOTE: TLS support
                #
                # the default-network-drivers() source driver opens the TLS
                # enabled ports as well, however without an actual key/cert
                # pair they will not operate and syslog-ng would display a
                # warning at startup.
                #
                #tls(key-file("/path/to/ssl-private-key") cert-file("/path/to/ssl-cert"))
                tls( key_file("/etc/syslog-ng/cert.d/serverkey.pem")
                cert_file("/etc/syslog-ng/cert.d/servercert.pem")
                ca_dir("/etc/syslog-ng/ca.d"))
        );
};
destination d_local {
        file("/var/log/messages-wso2.log");
        file("/var/log/messages-kv.log" template("$ISODATE $HOST $(format-welf --scope all-nv-pairs)\n") frac-digits(3));
};
log {
        source(s_network);
        destination(d_local);
};

Topic 3 : Enabling TLS encryption and mutual authentication

Task breakdown for topic 3 is as follows,
1. Creating self-signed certificates
1.1. Procedure — Creating a CA
1.2. Procedure — Creating a server certificate
1.3. Procedure — Creating a client certificate

2. Configuring Syslog-Ng OSE

2.1. Procedure — Configuring the syslog-ng OSE server

References used for Mutual authentication using TLS :

syslog-ng Open Source Edition 3.25 — Mutual authentication using TLS

Following are the configurations to be added to the syslog-ng.conf

syslog-ng.conf

Topic 4 : Setting Up Keystores for a Client and a Service in WSO2 EI

Create Client and Service Keys

1. Create the two sets of keys for the service and the client using the keytool (comes with the JDK).

keytool -genkey -alias client -keyalg RSA -keystore client-new.jks

2. You will be asked a series of questions. Fill in the respective details

3. The created keys are stored in the client-new.jks file which is a Java KeyStore under the alias client. You the below command to list the contents of the keystores.

keytool -list -v -keystore client-new.jks -storepass wso2client

4. Similarly create the service’s keys using the following command

keytool -genkey -alias service -keyalg RSA -keystore service-new.jks

5. Similar to above you can use the below command to list the contents of the keystore.

keytool -list -v -keystore service-new.jks -storepass wso2service

6. Since the keys have already been created in the topic 3. Remove the keys that was generated in the keystore by default. Use the following command to do this task:

keytool -delete -alias boguscert -storepass wso2keystore -keystore client-new.jks
keytool -delete -alias boguscert -storepass wso2keystore -keystore service-new.jks

7. The output produced in the client.pem and service.pem (topic 3) files are plain text. To import these signed certificates into the keystores, convert them into the binary (DER) format using openssl x509 command.

client :

openssl x509 -outform DER -in client.pem -out client.cert

service :

openssl x509 -outform DER -in service.pem -out service.cert

CA :

Convert the CA’s certificate to the binary form to be imported to both keystores.

openssl x509 -outform DER -in cacert.pem -out cacert.cert

8. Import the Certificates

8.1 Import the CA’s self-signed certificate to both client and service keystores. Use the alias ca to identify the CA’s certificate. The keytool will display the information in the certificate and will ask for confirmation to import. Type in “yes” and confirm the import

service :

keytool -import -file cacert.cert -keystore service-new.jks -storepass wso2service -alias ca

client :

keytool -import -file cacert.cert -keystore client-new.jks -storepass wso2client -alias ca

8.2 Import the self signed certs

Client :

keytool -import -file client.cert -keystore client-new.jks -storepass wso2client -alias client

Service :

keytool -import -file service.cert -keystore service-new.jks -storepass wso2service -alias service

In order to allow secure communication between the client and the service, make sure that each party has the others’ public key with them. Import the client.cert into the service’s keystore and the service.cert into the client’s keystore.

Since certificates added are signed by a trusted certificate, they will be simply imported to the keystore and the keytool.