Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Date

Authors

John Simmons (Deactivated) Adewale Adekoya Jayanta Saha

Status

Documenting

Summary

GMC sync for old & New reval were blocked by the GMC

Impact

Data between GMC and Reval apps was not in sync between 00:05 and 09:43

Non-technical Description

GMC sync for old reval didnt didn't work last night as were being blocked by the GMC

...

Trigger

Unknown change on GMC firewall

...

Detection

Slack notification received

...

on inspection of the logs we found this:

...

Resolution

emailed GMC who stated that they had made a security change overnight and that had made the service unavailable to us, they made a change on their end and asked us to rerun our job that was completed successfully.

...

Timeline

  • - 00:05 alert send by nightly job to Slack

  • - 08: 57 reran job to see if it was a temporary network error

  • - 09:10 Adewale Adekoya sent an email to GMC asking if the problem could be looked into

  • - 09:39 Peter McNair from the GMC replied to the email asking us to try again

  • - 09:43 ran the job again and it was completed successfully.

...

Root Cause(s)

Security change at GMC end caused the nightly ETL not to be able to connect to get the data needed.

...

Action Items

Action Items

Owner

...