Case Study: TIS-EsrReconciliationService

Overview

Upgrading TIS-EsrReconciliationService has proven to be a very difficult but informative process. Due to its sophisticated integration tests, reliance on third party dependencies and unitarization of a shared custom spring-boot starter, a number of issues arose during the attempted upgrade.

Below is a rough diagram showing the relationship between the service itself and our own custom spring boot starter.

TIS-rabbit-mongo-spring-boot-starter provides the following:

• Reusable configuration for MongoDB and RabbitMQ connections with the view of enabling consistent integration testing across multiple ESR Projects

• Automatic Bean configuration and creation for use in the integration tests

It should be quite clear therefore that upgrading the service necessitates upgrading the starter.

Issue 1: Java Version Upgrade

The particular headache for upgrading from 11 to 17 is JEP 403. This change enforces strong encapsulation of JDK internal classes, a list of which can be found here. What this means in practice is that (mis)use of reflection to access fields in these particular packages is now forbidden, and previous workarounds have been removed.

The challenges that this raises are:

• The error messages resulting from this change are fairly generic and give little indication of their true origin

• java.lang.reflect.InaccessibleObjectException: Unable to make protected java.lang.ClassLoader() accessible: module java.base does not "opens java.lang" to unnamed module

  • This means that it can be difficult/impossible to pin down exactly which dependency is causing the error to arise

  • The particular structure of this project, with the reliance on third parties and a separate custom spring boot starter, itself composed of several submodules, further compounds this issue.
    
    

• There is not a lot of widely available discussion around issues that arise from this

  • There are typically two solutions repeated everywhere:

    1. “Just downgrade to Java 11” - obviously not a real solution, but is the most common first answer

    2. Use the compiler argument --add-opens java.base/java.lang=ALL-UNNAMED(or similar) to allow reflective access

       • This is kind of a patch anyway rather than a real solution, as JEP 403 is to fix a security issue, so opening up the closed packages is just maintaining that issue

       • This doesn’t seem to work well with more complicated project structures like we have here and to some extent by the jenkins pipeline itself.

         • Short version: it’s a runtime issue, and having tested adding the compiler argument in multiple places, it never seemed to work

After a lot of investigation, dependency upgrades and trial and error - it was determined that the final source of this error was from the Spring Boot dependencies themselves. Upgrading to Spring Boot 3 was avoided for a long time as it would introduce further breaking changes (see below) and there was some (false) assurance from various sources that the latest versions for Spring Boot 2 should be fine with Java 17.

Issue 2: Spring Boot Upgrade

Once it was determined that it was the Spring Boot dependencies that were causing the issue, an upgrade to Spring Boot 3 was attempted.

This immediately solved the java.lang.reflect.InaccessibleObjectException: Unable to make protected java.lang.ClassLoader() accessible: module java.base does not "opens java.lang" to unnamed module error, but replaced it with a new set of issues.

However, the new set of issues are more of a refactor than a troubleshoot, which is a much better position to be in.

More in-depth description

Way forwards: Upgrade Custom Starter → Upgrade Services

Description of autoconfiguration deprecation

Description of potential refactor points

More Diagrams more good