Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Next »

Date

Authors

Rob Pink Steven Howard

Status

Summary

Users experiencing issues with TIS - crashing and slow loading over a couple of days

Impact

Users unable to use TIS
TIS21-6610 - Getting issue details... STATUS

Non-technical Description

Users on TIS reported that the application was very slow and occasionally crashing when attempting to retrieve or update data. This was the result of proxy errors being returned from the TCS enpoints. image-20241010-095351.png


Trigger


Detection


Resolution


Timeline

All times BST unless otherwise indicated.

  • 1229 user reported problem on Teams “TIS being slow / crashing when doing updates - A few of us in Wessex are experiencing issues with TIS when trying to do updates - it keeps crashing and is being really slow loading over the past couple of days.”

  • 1300 Message acknowledged

  • 1342 Errors spotted on esr-sentry channel so ESR services stopped to prevent potential errors

  • 1353 Service restarted and reported to users that looks resolved

  • 1410 Users confirmed things back to normal

  • 16:00 Logs relating to ESR checked and ESR services restarted

  • 11:00 Applicant records from 8th Oct sent out

  • 12:00 Notification records from 8th Oct sent out excluding stale

5 Whys (or other analysis of Root Cause)

Q. Why was TIS responding slowly or crashing?
A. TCS endpoints were not returning data and timing out with proxy errors

Q. Why was TCS erroring and not returning data
A. ??

Q. Why were there errors on sentry-esr?
A. TCS was unresponsive


Action Items

Action Items

Owner

See also:


Lessons Learned

  • No labels