Date |
|
Authors |
|
Status | Documenting TIS21-4706 - Getting issue details... STATUS |
Summary | |
Impact |
Non-technical Description
Trigger
When a Reval admin submits a Recommendation, a success message is shown in the UI followed by an error, but not submission is made to GMC
Detection
User reported issue on teams
Resolution
Timeline
All times in BST unless indicated
09:16 : User reports on Teams
I have tried to make a revalidation recommendation in TIS this morning and got 'An error occurred! - Please try again'. i have checked GMC Connect and it looks like it has gone though, can you advise if there are any issues.10:15 : User contacted requesting GMC number
11:19 : Call with user to demo the issue
11:49 : Huddle to debug and understand if isolated or system wide
13:00 : Identify that requests to profile service from integration service are not appearing in logs so redeployed integration service
13:10 : Call with user to test. Still broken.
13:30 : Other admins confirm they are also experiencing the same issue.
15:30 : Checked API-Gateway logs and found no logs for submitting recommendations on Monday.
16:00 : Found there were lots of timeouts in
prod-revalidation-api-gateway-authoriser, then updated it from 3s to 10s.19:30 : Updated the version of
nodeto run Lambdaprod-revalidation-api-gateway-authoriserfrom 12 to 16.08:45 : Checked
awslogs-prod-tis-revalidation-recommendationslogs for "submitting request to gmc for recommendation:" which were showing this morning09:00 : Verified with users that recommendations were being submitted.
5 Whys (or other analysis of Root Cause)
Action Items
Action Items | Owner | Comments |
|---|---|---|
Add Comment