Versions Compared

Old Version 3

changes.mady.by.user David Hay (Unlicensed)

Saved on

compared with

New Version Current

changes.mady.by.user David Hay (Unlicensed)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. call setChannelTransacted(true) on the RabbitTemplate Spring Bean.

  2. register a org.springframework.amqp.rabbit.transaction.RabbitTransactionManager Spring Bean

  3. add the org.springframework.transaction.annotation.Transactional Annotations to the code which we want to wrap in a Rabbit Transaction. This Transactional Annotation works for both Mongo and Rabbit Transactions.

Transactions in MongoDB

Transactions are a relatively new feature of MongoDB (since v4.0)

...

Simply put, a MongoDb ReplicaSet is a cluster in primary/secondary mode. All writes are sent to the master primary and then the data is propagated to the secondaries. You can setup a MongoDB ReplicaSet with just one node - this is good for testing Transactions but a single node ReplicaSet is not a good idea for production use cases.

...

  1. register a org.springframework.data.mongo.MongoTransactionManager Spring Bean.

  2. add the org.springframework.transaction.annotation.Transactional annotations Annotations to the code which we want to wrap in a MongoDB Transaction. This Transactional Annotation works for both Mongo and Rabbit Transactions.

  3. make sure any Mongo Collections (Collections are a similar to Tables in a typical RDBMS) you need to use within a Transaction are created outside of the Transaction, in advance. When using Mongo Transactions - you can no longer create a collection ‘on demand’ within a transaction by simply inserting into it. To work around this there is a class com.hee.tis.esr.common.transactions.MongoCollectionInitializerConfig which looks for instances of Spring Data Mongo Repositories and creates any Mongo Collections required on application startup. This is a temporary solution until we are using something like mongobee to make sure Collections are created as part of the Microservice Installation startup process.

The MongoTransactionManager works with inserts/updates to mongo through Spring Mongo Data repositoriesRepositories.

There are not a great number of examples showing how Spring works with MongoDB Transactions. The example here is a bit limited.
https://github.com/spring-projects/spring-data-examples/tree/master/mongodb/transactions

It relies on custom embeddedMongoDb for testing and seems to suggest you need to use mongoTemplate when doing inserts/updates to Mongo within a Transaction. This is misleading - you can use Spring Data Mongo Repositories which are easier to use than mongoTemplates.

Global Parent Transaction Manager

We have linked the RabbitMQ and MongoDB Transaction Managers so that they work together. To do this we create a parent/global transaction manager which delegates its commit & rollback calls to its child (Mongo and Rabbit) Transaction Managers. We use the Spring provided ChainedTransactionManager which must have 1 or more child transaction managers ( You can’t use ChainedTransactionManager without any child transaction managers). The ChainedTransactionManager is registered as a Primary TransactionManager so Spring knows which of the Transaction Managers to use when there is more than 1.

...

If have at least 1 Rabbit or Mongo Transaction Manager, then we create a ‘parent‘Parent' transaction manager.

If we have neither Rabbit or Mongo transaction manager - we register a dummy/NoOp - TransactionManager. If we have code annotated with ‘@Transactional’, Spring demands that we have a Transaction Manager - so we register a dummy, NoOp TransactionManager to keep it happy. We cannot switch off the ‘@Transactional' annotation, so we just use a dummy Transaction Manager.

...

Common Classes in the com.hee.tis.esr.common.transactions package, can be used to support bring Rabbit and Mongo Transactions to ESR Microservices in a standard, testable way. The intention is that these common classes end up in a common shared library - so the code is only in once place.

#

Class Name

Description

1

ImportTransactionConfig

A Spring Annotation used to pull in the Transaction Support.

ESR Microservices wanting to pull in TransactionSupport should add the @ImportTransactionConfig annotation to their Spring Boot “Application” class.

2

MongoCollectionInitializerConfig

Looks for Spring Data Mongo Repositories on startup and creates the required Collections in MongoDB if they don’t already exist.

3

NoopTransactionManager

Noop/Dummy Transaction Manager class. SeeTxConfigNoop

4

TxConfig

On application startup, logs the actual TransactionManager being used. This is just to help with debugging.

5

TxConfigGlobal

Creates a ChainedTransactionManager parent to wrap the Rabbit and/or Mongo Transaction Managers, if required. If created, this Spring Bean is named txManagerGlobal.

6

TxConfigMongo

Creates a MongoTransactionManager Spring Bean if the property app.transactions.mongo.enabled has value 'true’.

If created, this Spring Bean is named txManagerMongo

7

TxConfigNoop

Creates a Spring Bean using NoopTransactionManager if there is no Bean named txManagerGlobal. If created, this Spring Bean is named txManagerNoop

8

TxConfigRabbit

If the property app.transactions.rabbit.enabled has value 'true’ do the following:

  • Create the RabbitTransactionManager Spring Bean namedtxManagerRabbit

    • call setChannelTransacted(true) on the RabbitTemplate.

9

TxConstants

Contains public constant values related to this package. Things like Spring Bean names and important property names.

Testing Transaction Support

We have done some work to allow us to test Transactions that apply to both Rabbit and MongoDB using Test Containers. By using Test Containers we use actual instances of MongoDB and RabbitMQ (albeit provided by Docker Containers) in our tests. This is useful because we are using actual instances not ‘test’ instances which might behave different from the ‘real' thing.

The base code for testing transactions is within the following packages

  • com.hee.tis.esr.common.transactions.test

#

Class

Description

1

BaseRabbitAndMongoTxServiceTest

This abstract class has some tests which check that Rabbit and Mongo Transactions are setup correctly in a Spring Boot Application/ESR Microservice. It relies on the superclass having a ‘@SpringBootTest’ which refers to the Spring Boot Application (ESR microservice ) that you wish to check has Rabbit and Mongo Transactions enabled and configured correctly. The test code is common but the Configuration of the Spring Boot Application will be specific to each ESR microservice. To create an instance of this test

  1. extend BaseRabbitAndMongoTxSerfviceTest

  2. use @SpringBootTest

  3. use @LocalTransactionSupport

  4. use@EnableMongoRepositories(basePackageClasses'=<YourSpringBootApplication>.class)

  5. @UseTransactionalTestContainers
    (or some other way of pulling in a Mongodb and RabbitMQ instance for testing)

Instances of this test create their own Rabbit and Mongo Resources to check that each Spring Boot application has transactions correctly configured. The test resources are

  1. Mongodb collection named “txmessage“

  2. RabbitMQ queue named “tx.test.queue“

  3. Spring Bean named “tx.recvd“ to hold the messages received from “tx.test.queue“.

  4. Spring Data Mongo Repository named TxMessageRepository for accessing mongo collection “txmessage“ from Spring/Java.


A concrete example of this can be found in the ESR Data Export service - the class is com.hee.tis.esr.esrdataexport.transaction.RabbitAndMongoTxServiceTest

2

LoadTransactionSupport

A Spring annotation which Enables Mongo Repositories and loads the Spring Config from TransactionTestConfig

3

TransactionTestConfig

Test Spring Bean Configuration. Creates a test queue, sets up a RabbitMQ listener which will put any messages received from the test queue into another Spring Bean that is a simple list of Strings (for testing).

4

TxMessage

A POJO which maps to the Test Mongo Collection called txmessage.

5

TxMessageRepository

A Test Spring Data Mongo Repository interface for the TxMessage POJO & txmessage collection.

6

TxRabbitAndMongoService

A Test Spring Service which is used to test Rabbit and Mongo Transactions. The process method sends RabbitMQ messages and updates MongoDB and can conditionally throw Exceptions. The intention is that if we are within a transaction and throw an exception, the previous message sending and mongo update are rolled back.

7

TxTestConstants

Contains public constants like test queue names, this package name ( for Spring Scanning), Test Spring Bean names, the Spring profile name used to provision a Test Container of a MongoDb instance with ReplicaSet support.

  • com.hee.tis.esr.common.autoconfigure

This package contains the class 'EmbeddedMongodbReplicaSetBootstrapConfiguration'. This class it loaded via Spring AutoConfiguration. Spring knows to load it via AutoConfiguration because its referred to in the smallsrc/test/resources/META-INF/spring.factories config file.

Code Block
org.springframework.cloud.bootstrap.BootstrapConfiguration=\
com.hee.tis.esr.common.autoconfigure.EmbeddedMongodbReplicaSetBootstrapConfiguration

The EmbeddedMongodbReplicaSetBootstrapConfigurationis only enabled if the test property 'embedded.mongodb.rs.enabled' has value true. This property is set in the test configuration file “bootstrap-replicaset.properties“ which is only read when the Spring profile “replicaset” is active.

There are complex dependencies between spring profiles, properties, beans and autoconfigured classes which create test containers and additional spring properties. The Spring Boot Test Container dependencies from “com.playtika.testcontainers.*" andEmbeddedMongodbReplicaSetBootstrapConfiguration” depend on the Bootstrapping feature of Spring Cloud.

The EmbeddedMongodbReplicaSetBootstrapConfigurationis processed by the Spring Cloud Bootstrapping feature which is pulled in via the gradle test dependency “org.springframework.cloud:spring-cloud-starter“.

This EmbeddedMongodbReplicaSetBootstrapConfiguration can startup a MongoDB with ReplicaSet mode in a Test Container. It can also set the following properties.

Code Block
embedded.mongodb.port
embedded.mongodb.host
embedded.mongodb.database
embedded.mongodb.replicaset

These properties can then be used by the test code to build up a MongoDB URI and connect to MongoDb within the test container. See the test configuration file “application-replicaset.yml“.

...

We had to write custom code in “EmbeddedMongodbReplicaSetBootstrapConfigurationis“ to setup MongoDB in a TestContainer in ReplicaSet mode so we can test transactions. Our custom code is based on the classcom.playtika.test.mongodb.EmbeddedMongodbBootstrapConfiguration from the library “com.playtika.testcontainers:embedded-mongodb:1.32“ - this starts MongoDB within a Test Container but not in ReplicaSet mode.