Java 17 support

[dladlk]

On an Oxalis Community Technical Dialog Meeting November 24, 2021 it was discussed whether Oxalis can/should switch/support Java 17.

At the moment Oxalis works on Java 8, and is not expected to work on Java 11 because of big clenaup, done in JEP 320:

https://www.oracle.com/java/technologies/javase/11-relnote-issues.html#JDK-8190378

Fast research shows:

• Migration requires not only source code changes, but also changes in referenced dependencies
• Some dependencies are already parts of OxalisCommunity - e.g. vefa-peppol libraries
• But vefa-peppol depends on not migrated to OxalisCommunity DIFI repo projects - especially efm-certvalidator and efm-commons-libs

To support Java 17, the process could consider:

• removed classes from Java since version 11 and future versions
• more strict security (e.g. XML Digital Signature rejects RSA-SHA1 algorithm by default at least since Java 14)
• new java.lang. classes, e.g. Record - which could conflict on compilation with asterisk imports
• more

As a very early proof of concept, draft migration was tried for vefa-peppol project with next change notes:

• Bump jacoco-maven-plugin from 0.8.1 to 0.8.7 which supports java 17 (since 04.05.2021, experimental, see https://www.eclemma.org/jacoco/trunk/doc/changes.html )
• Bump lombok from 1.18.12 to 1.18.22 which supports java 17 (since 06.11.2021, see https://projectlombok.org/changelog )
• Migrate JAXB from javax.xml.bind (removed from Java 11, see https://www.oracle.com/java/technologies/javase/11-relnote-issues.html#JDK-8190378 ) to jakarta.xml.bind (see https://github.com/eclipse-ee4j/jaxb-ri/releases/tag/3.0.0-RI ): rename packages, upgrade dependencies, JAXB target in pom and namespace in evidence.xjb, see https://javaee.github.io/jaxb-v2/doc/user-guide/ch03.html#deployment-maven-coordinates with changes:
  -- jaxb2-maven-plugin from 1.6 to 2.5.0,
  -- target from 2.1 to 3.0,
  -- namespace in *.xjb from xmlns:jaxb="http://java.sun.com/xml/ns/jaxb" to xmlns:jaxb="https://jakarta.ee/xml/ns/jaxb" and jaxb:version="2.0" to jaxb:version="3.0"
• BdxlLocator: Avoid wildcard imports: since Java 17, org.xbill.DNS.Record conflicts with java.lang.Record if no explicit import defined
• XmldsigVerifier: TEMPORARY!!! Disable secure validation via setting org.jcp.xml.dsig.secureValidation to FALSE - but only because test files are signed with SHA-1... Error: "javax.xml.crypto It is forbidden to use algorithm http://www.w3.org/2000/09/xmldsig#rsa-sha1 when secure validation is enabled". See Java 14 XML Signature Secure Validation Mode
  In OpenJDK 17 it is enabled by default: https://bugs.openjdk.java.net/browse/JDK-8259801

As vefa-peppol depends on DIFI libraries, similar draft migration was done on /efm-certvalidator (successfully) and /efm-commons-libs (paused, stucked on stubs generation for WSDL at commons-busdox, where tricky Maven goals are defined ( wsimport1 and wsimport2) - but it looks possible to fix.

In general, for now it looks doable to have a single version of Oxalis, able to run starting from Java 8 up to at least Java 17: keep compilation level on java 8, but make changes in the code and dependencies.

Theoretically, it could be possible to keep javax.xml.bind package and use it as dependency, one of the worries is in potential differences in interfaces, specific for some JDK implementations, and in libraries used to compile JAXB stubs. But in this case less changes are required in external libraries.

One of questions to clarify - whether DIFI is going to modify no.difi.commons.commons-XXX libraries (used in Oxalis) or is able to accept, review and test pull requests to make changes there. Alternative - migrate them to OxalisCommunity repository if it was missed by accident :)

[dladlk]

Based on voting results of the last Oxalis Technical Meeting March 30, it could be considered not to migrate the whole project to be able to compile with JDK 17, but instead only to support it running on JDK 17, at least as an intermediate step before final migration.

This can be achieved with much lower efforts, and even done by users of the Oxalis themselves:

1. It is quite easy to include into the project external libraries, removed in Java 11 - most javax packages have separate maven libraries.

2. Bump to newer version of Guice - 5.1.0, which supports JDK 17 (current version is 4.2.2).

3. Oxalis lookup feature (both BDXL and Busdox) uses dnsjava-2.1.9 (org.xbill.DNS package) - which tries to load sun.net.dns.ResolverConfiguration (on Windows) to get list of DNS servers, and without changes fails with error

java.lang.IllegalAccessException: class org.xbill.DNS.ResolverConfig cannot access class sun.net.dns.ResolverConfiguration (in module java.base) because module java.base does not export sun.net.dns to unnamed module

This can be solved by starting Oxalis with additional VM argument --add-exports java.base/sun.net.dns=ALL-UNNAMED

Above changes can be done without modifying any line in Oxalis project, e.g. by importing oxalis-commons, oxalis-api, oxalis-outbound and oxalis-as4 to a a new project and adding next dependencies to add required javax.* implementations and overwrite Guice version:

<!-- START: Support JDK 17 -->

<!-- Overwrite dependency from oxalis-commons with 4.2.2 by 5.1.0 to support Java 17 -->
<dependency>
	<groupId>com.google.inject</groupId>
	<artifactId>guice</artifactId>
	<version>5.1.0</version>
</dependency>

<!-- javax.activation deprecated in 9 and removed in 11 - see https://openjdk.java.net/jeps/320 -->
<dependency>
	<groupId>javax.activation</groupId>
	<artifactId>javax.activation-api</artifactId>
	<version>1.2.0</version>
</dependency>

<!-- javax.xml.bind deprecated in 9 and removed in 11 - see https://openjdk.java.net/jeps/320 -->
<dependency>
	<groupId>jakarta.xml.bind</groupId>
	<artifactId>jakarta.xml.bind-api</artifactId>
	<version>2.3.3</version>
</dependency>
<dependency>
	<groupId>com.sun.xml.bind</groupId>
	<artifactId>jaxb-impl</artifactId>
	<version>2.3.3</version>
	<scope>runtime</scope>
</dependency>

<!-- JavaSE 9 moved package javax.xml.soap to the module java.xml.ws -->
<dependency>
	<groupId>javax.xml.soap</groupId>
	<artifactId>javax.xml.soap-api</artifactId>
	<version>1.4.0</version>
</dependency>
<dependency>
	<groupId>javax.xml.ws</groupId>
	<artifactId>jaxws-api</artifactId>
	<version>2.3.1</version>
</dependency>

<!-- Java JWS and SAAJ -->
<dependency>
	<groupId>javax.jws</groupId>
	<artifactId>javax.jws-api</artifactId>
	<version>1.1</version>
</dependency>
<dependency>
	<groupId>com.sun.xml.messaging.saaj</groupId>
	<artifactId>saaj-impl</artifactId>
	<version>1.5.3</version>
</dependency>

<!-- END: Support JDK 17 -->

This was successfully tested with AS4 sending with both BDXL and BusDox lookup. Testing inbound server modifications for receiving testing are pending.

BTW, some unit tests from oxalis-commons/oxalis-outbound under JDK 17 where lookup is done, fail with error "Unable to verify document signature.": "javax.xml.crypto.MarshalException: It is forbidden to use algorithm http://www.w3.org/2000/09/xmldsig#rsa-sha1 when secure validation is enabled".

This happens because in JDK 17 by default a strict security is activated which forbids SHA1 based signature algorithms, still used in many SMP implementations - see JDK17/conf/security/java.security file:

jdk.xml.dsig.secureValidationPolicy=\
    disallowAlg http://www.w3.org/TR/1999/REC-xslt-19991116,\
    disallowAlg http://www.w3.org/2001/04/xmldsig-more#rsa-md5,\
    disallowAlg http://www.w3.org/2001/04/xmldsig-more#hmac-md5,\
    disallowAlg http://www.w3.org/2001/04/xmldsig-more#md5,\
    disallowAlg http://www.w3.org/2000/09/xmldsig#sha1,\
    disallowAlg http://www.w3.org/2000/09/xmldsig#dsa-sha1,\
    disallowAlg http://www.w3.org/2000/09/xmldsig#rsa-sha1,\
    disallowAlg http://www.w3.org/2007/05/xmldsig-more#sha1-rsa-MGF1,\
    disallowAlg http://www.w3.org/2001/04/xmldsig-more#ecdsa-sha1,\
    maxTransforms 5,\
    maxReferences 30,\
    disallowReferenceUriSchemes file http https,\
    minKeySize RSA 1024,\
    minKeySize DSA 1024,\
    minKeySize EC 224,\
    noDuplicateIds,\
    noRetrievalMethodLoops

Instead of modifying this file by removing from there

    disallowAlg http://www.w3.org/2000/09/xmldsig#sha1,\
    disallowAlg http://www.w3.org/2000/09/xmldsig#rsa-sha1,\

unit tests can be fixed by adding org.apache.wss4j.common.crypto.WSProviderConfig.init(); which adds XMLDigSiRI from xmlsec-2.1.5 (org.apache.jcp.xml.dsig) instead of Java 17 xml security (org.jcp.xml.dsig) - so it works exactly as in case of AS4 Module usage, where it is invoked in the normal flow.

One more notice - at least locally, when there are many DNS lookup servers configured, SMP lookup time can be very big (16 seconds!), because internally dnsjava can query each of them one by one until gets response. This can be debugged by calling org.xbill.DNS.Options.set("verbose", "true"); - it will show list of DNS server IPs and lookup details. To overcome it, system property -Ddns.server=8.8.8.8 can be passed to optimize performance to give preferred DNS server to start with (see more https://github.com/dnsjava/dnsjava )

[dladlk]

Done successful testing of above mentioned dependencies with Oxalis AS4 server side.

Tests by eu.sendregning.oxalis.Main of client and server on Java 17 did not show big difference in performance comparing to equal Java 8 setup.

BTW, these dependencies can be integrated into Oxalis similar to CXF support of Java 9 and further via profiles, see example:
https://github.com/apache/cxf/blob/842272447a054cf89a2ed7da0d100fc05a0508ae/parent/pom.xml#L2302

Additionally, some heavy transitive dependencies like jacorb-omgapi and jboss-rmi-api_1.0_spec from cxf modules can be to exclude to reduce size of standalone jar or war packaging.

[aaron-kumar]

Thanks a lot @dladlk 👏 for such a extensive research and contribution.
You successfully described whatever required for Java 17 upgrade during Phase 1(running on Java 17) and Phase 2 (support Java 17 features/compile support).

Is it ok for you to document the steps in Wiki pages?:

• Oxalis (Core & AS2): https://github.com/OxalisCommunity/oxalis/wiki/Oxalis-Java-17-Support
• Oxalis-As4: https://github.com/OxalisCommunity/Oxalis-AS4/wiki/Oxalis-Java-17-Support
• Vefa-peppol : https://github.com/OxalisCommunity/vefa-peppol/wiki/Oxalis-Java-17-Support

[dladlk]

Both Oxalis and Oxalis-AS4 projects are updated in the branch feature/java-17 with latest changes from master and new version 6.0.0-SNAPSHOT

Github actions were updated to build both projects with maven on java 17 and publish maven artifacts to OSS Sonatype snapshot repository, also as generate Docker images (at the moment limited only to amd64 to speed up the building process) with version latest-17.

Links:

• https://hub.docker.com/r/norstella/oxalis/tags?page=1&name=latest-17
• https://hub.docker.com/r/norstella/oxalis-as4/tags?page=1&name=latest-17
• https://oss.sonatype.org/content/repositories/snapshots/network/oxalis/oxalis/6.0.0-SNAPSHOT/
• https://oss.sonatype.org/content/repositories/snapshots/network/oxalis/oxalis-as4/6.0.0-SNAPSHOT/

Again, at the moment I see no problems to release the same maven artifacts for both java 8 and java 17 (except distribution package) and just 2 sets of Docker images - based on Java 8 and Java 17 - per each architecture - that is why old Dockerfile based on java-8 is kept in the repos, just one more Dockerfile-Java-17 was added.

[HegeErdal]

Is there any updates/plans for when version 6.0.0 (Oxalis with java17) will be released?