Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

Date

Authors

John Simmons (Deactivated) Adewale Adekoya Jayanta Saha

Status

Implementing

Summary

Legacy GMC sync failed and Connections showing an error

Impact

Data in Reval out of date and Admins unable to use Connections

Non-technical Description

GMC sync failed


Trigger

Unknown change on GMC firewall


Detection

Slack notification received

on inspection of the logs we found this:


Resolution

TODO: .


Timeline

  • - 00:05 alert send by nightly job to Slack

  • - 09: 50 reran job to see if it was a temporary network error. Adewale Adekoya sent an email to GMC asking if the problem could be looked into

  • - Began investigating with GMC.

  • - 00:05 GMC-Sync succeeded

  • - 09:20 Re-attempted changing connetions but this failed

  • - Issue narrowed down to an IP address but this hasn’t resolved the issue

  • - Further debugging info has been exchanged

  • - A further fix did not resolve the issue, and

  • Dec 9, 2021 - Sync job ran successfully however no updates from GMC on any changes made on their side

  • Dec 9, 2021 - Adewale Adekoya further enquires for any updates from the GMC but no response received


Root Cause(s)

  • Alerting from Slack (GMC Sync) & on MS Teams (Connections failure)

  • Logs show failed Web Service requests


Action Items

Action Items

Owner

Make it simpler to re-run the synchronised job. https://hee-tis.atlassian.net/browse/TIS21-2437

Joseph (Pepe) Kelly & John Simmons (Deactivated)


Lessons Learned

  • No labels