Non-technical Description

GMC was blocking our connection to their prod. environment somewhere.

 What was the issue?

The API issue was coming from their cloudflare infrastructure, a quick google of the error states: “Cloudflare Error 1020: Access Denied indicates that you’ve violated a firewall rule and your connection request has been blocked” This code (1020) was not on the list of GMC codes.

After contacting GMC, we got to know that GMC implemented some additional security on 20/07/2021 due to some nefarious activity that was hitting their service.  That mitigation blocked our connection to the API. On 21/07/2021 at 11am, GMC added a whitelist to that mitigation and that resolved the problem.

Trigger

Detection

Resolution

Timeline

21/07/2021 at 10:26 Katy raised issue

21/07/2021 at 10:26 John raised issue on Slack

21/07/2021 at 11:33 Ade raised issue with GMC

21/07/2021 at 11:55 GMC requested for more details

21/07.2021 at 12: 35 Ade supplied more details

21/07/2021 at 14:16 GMC emailed issue resolved

Root Cause(s)

Lessons Learned