Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

Date

Authors

Joseph (Pepe) Kelly

Status

Documenting

Summary

The database used for holding information sent to & received from ESR was unavailable. This meant data was unable to be processed.

https://hee-tis.atlassian.net/browse/TIS21-1980

Impact

Some applicant .

Non-technical Description

The database got very busy and wasn’t available to process changes from TIS.

Trigger

Under review

Detection

  • Application Error Alerting

Resolution

  • Restarted VM

  • Checking Dead Letter Queue.

Timeline

  • - 15:52 - First alert messages received

  • - 18:01 - Server restarted

  • - -

Root Cause(s)

Action Items

Action Items

Owner

Add more resilience because moving to a managed service was more complex than anticipated

https://hee-tis.atlassian.net/browse/TIS21-489

Review Applicants & Notifications that might need to be generated

Joseph (Pepe) Kelly Most dropped events were for placements that have started and so were disregarded as they would not have produced applicant records eligible for sending to ESR. We also found:

  • 1 record that had recovered, creating the necessary APP record

  • 1 record that a notification (to send Feb '22) that was

  • New post hasn’t been notified to trusts: MER/REM21/006/HT/001. We will ask the regional lead if Trusts can be contacted manually

  • There were additional unconfigured attempts for some notifications, e.g. correlation id: 6e004633-6e34-426b-94b3-69b2adfbb51c

Lessons Learned

  • No labels

Slack: https://hee-nhs-tis.slack.com/
Jira issues: https://hee-tis.atlassian.net/issues/?filter=14213