...
All times in BST unless indicated
: 08:53 : User reported (on Teams) revalidation module is showing one person under notice
: 08:53 : User reported (on Teams) revalidation module is showing one person under notice
: 08:53 : User reported (on Teams) revalidation module is showing one person under notice
09:30 : Stopped the 2-hourly checks of submitted recommendation, shortly after stopped the service temporarily to stop unhelpful logging
: 09:41 : Moved sync start messages to new queues for debugging
: 09:43 : Found logging to suggest incident started at 00:05 - around the time of the gmc sync job starting
: ~ 9:45 : Stopped gmc-client task on prod
: ~10:00 : Restarted gmc-client task on prod, observed the same issue, task stopped again.
: 10:15 : Changed Log level for gmc-client (set to debug) and pushed to preprod
11:30 added
JAVA_TOOL_OPTIONS
in task definition, then updated memory from 512M to 2G.
Root Cause(s)
...
Action Items
...