https://hee-tis.atlassian.net/browse/TISNEW-5495

• Root Cause(s)

• Trigger

• Resolution

• Detection

• Action Items

• Timeline

Root Cause(s)

• AWS Migration - Whitelisting IPs with GMC

• AWS Migration - authentication details - using old transform account

Trigger

• AWS migration

• User feedback

Resolution

• Requested GMC to whitelist IPs

• workaround - scheduled job running everyday to pull data

• Updated authentication details

Detection

• Reported on Teams Sep 24, 2020

  Open

  

   

Timeline

• Sep 24, 2020 - 08:10

• Sep 24, 2020 - 08:30 started to look into it

• Sep 24, 2020 - 08:43 AM - email sent to GMC requesting whitelisting IPs

• Sep 24, 2020 - 09:00 AM - started a data copy from azure to AWS to check PoC

• Sep 24, 2020 - 11:30 AM - Completed PoC and wrote a script to do the daily copy until the new IP’s are whitelisted

• Sep 28, 2020 - 08:30 AM - missing data, suspected that another process interrupted the sync, rerun the job and all is well.

• Sep 28, 2020 - 10:45 AM - changed script run times.

• Sep 30, 2020 - 2:13 pm - Confirmation from GMC that new IPs requested are whitelisted

• Sep 30, 2020 - 3:30 pm - Phil rang the GMC to chase messages sent to the ISServiceManager@gmc-uk.org 

• Oct 5, 2020 - 10@30 - User on teams confined working

• Open

  

   

Action Items

Lessons Learned

• Brain drain from the team - a big risk not having many team members know how existing reval works

• 3rd Parties turnaround time

• Get new reval live