Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

Date

Authors

Status

In process

Summary

The requests for a full data sync for new trainee profiles were not being processed, resulting in missing data (e.g. email address) in trainee profiles.

Impact

Onboarding emails were not being sent (recorded as ‘FAILED’) since TSS had no record of some new trainee email addresses.

Non-technical Description

When a trainee profile is created for the first time, TSS sends a “DataRequest” message to TIS to trigger a full data sync for the matching Person record. However, the TSS sync service has been failing to process those messages since August 2024. As a consequence, these new profiles may be missing key information that is needed to populate and send onboarding notifications, e.g. trainee email address, since TSS would otherwise only get this information if it happened to be updated in TIS after the trainee profile was created.

A large number of recent trainee onboarding notifications are recorded as ‘failed’ due to a missing email address.


Trigger

A version update of a library component used to send and receive messages (spring-cloud-aws upgraded from 2.4.x to 3.1.x, which we implemented in Dec 2023) introduced a deserialization message attribute with a class from the tis-trainee-details service (JavaType=uk.nhs.hee.trainee.details.event.ProfileCreateEvent)which the SQS listener in tis-trainee-sync then unsuccessfully attempted to use to deserialize the message, since it does not have access to that class.

As there is no DLQ for the profile created queue, no AWS alarms exist and the Sentry error-reporting service appears to be misconfigured for the sync service. As a consequence the error remained undetected.


Detection

Issue of large numbers of ‘failed’ onboarding emails reported to TSS team 7 Nov 2024.


Resolution


Timeline

All times GMT unless otherwise indicated.

  • 10:30 As part of the discussion around TIS21-6673 - Getting issue details... STATUS , it was mentioned that unusually large numbers of emails for trainee onboarding were being recorded as ‘failed’ due to no email address, but that the trainees in question had email addresses in TIS.



Action Items

Action Items

Owner

See also:


Lessons Learned

  • No labels

0 Comments

You are not logged in. Any changes you make will be marked as anonymous. You may want to Log In if you already have an account.