ESR Integration Details

The ESR integration consists mostly from a modern eventful tech stack.

It is composed of the following:

• Java 11

• Spring boot 2.2

• Gradle

• TestContainers

• RabbitMQ

• MongoDB

• AWS (Lamda & S3)

• Jenkins CICD

• Containers for building

The following is a description of each part of the ESR integration

Reconciliation

Services involved:

• InboundDataReader

• Reconciliation

• TCS

This part of the system retrieves a rest request from AWS upon a file upload to the FTP server (S3). The InboundDataRead receives this request and downloads the newly upload file.

Once downloaded, the file is then split into POR/POS/ASG records and each of those types are then sent to the esr.queue.<TYPE>.split queues.

The reconciliation service then picks up the POR/POS types, saves them to the database and sends it to another queue esr.queue.<TYPE>.saved

The same service then picks up the message and tries to find the relating post in TCS by making a rest request. If it exists, the post id is then saved along side the position and a new message is placed on the esr.queue.position.reconciled queue for the AppRecord and Notification services to pick up and process

AppRecords

Services involved:

• AppGenerator

• DataExporter

• TCS

When an ESR Position has been reconciled against a TIS Post, AppRecords are then generated. For every Placement linked to said Post, we’ll generate an AppRecord and send it to the DataExport service to persist.

The DataExporter when receiving the AppRecords via the esr.queue.apprecord.generated queue, will check if they already exist. If it does exist, an update will occur if the data has not yet been exported to ESR, otherwise it’ll drop the AppRecord.

AppRecords that then match a criteria (within a window of +2 days and less than 13 weeks start date) are then picked up, written to file which is then FTP’d to the AWS server

Notifications

Services involved:

• NotificationGenerator

• DataExporter

• TCS

Also after the reconciliation of Posts and Positions, the notification service then starts to generate Notification records. Like the AppRecords, they will be generated and sent to the DataExporter where they will be saved if they have not been submitted before. Also like the AppGen process, notifications are picked up are a certain time if they meet the criteria before being written to file and sent

Inbound

Services involved:

• InboundDataWriter

• TCS

The inbound stream takes data ASG data from the RMT files and updates the relating data in TIS, this data is pushed from the splitter to the esr.queue.asg.split queue and is read by the InboundDataWriter service. Currently this data is limited to Personal data such as personal details, address and absence and is updated via REST to TCS using POST, PUT and PATCH endpoints.

Confirmations

not yet been defined

All services have been defined with transaction support, this means that if an operation fails with MongoDB or RabbitMQ, any previous actions done with those within the same transaction will be rollback and can be attempted again