Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

Date

Authors

Status

Documenting

Summary

DMS tasks have been down for a couple of months (since mid October) and were brought back up only today

Impact

No data changes synchronized over to TSS in those two months

Non-technical Description

  • DMS was not running on either prod and preprod. Changes in the TIS databases meant to be captured and synchronized to TSS must not have been captured while DMS was down.


Trigger

  • Mysql server not being updated to anticipate DMS’s change of address


Detection

  • Andy Dingley noticed the tasks were not running on both preprod and prod


Resolution

  • Whitelisting of DMS’s new address on the Mysql server


Timeline

  • (approximate) - DMS tasks stop working

  • 14:10 GMT - Andy finds the the DMS tasks are not running

  • 15:30 GMT - Tasks restarted successfully after the whitelisting of DMS’s address


Root Cause(s)

  • DMS’s address change


Action Items

Action Items

Owner

Add monitoring to DMS

https://hee-tis.atlassian.net/browse/TIS21-2443

Mitigate to prevent this from happening in the future


Lessons Learned

  • No labels