Non-technical Description

Bulk upload of files on HEE Prod failed to upload, displaying an Unknown Server Error.

Trigger

• KeyCloak issue - once fixed, Bulk upload appeared to work.

Detection

• User notification on Teams.

• 

• Errors on the Bulk Upload page.

  

• Error after successful completion of the upload job, as stated on the Export errors excel file

  

I/O error on PATCH request for "http://tcs:8093/tcs/api/bulk-people/": {  "timestamp" : "2021-03-18T14:18:03.153+00:00",  "status" : 500,  "error" : "Internal Server Error",  "message" : "",  "path" : "/tcs/api/bulk-people/"}; nested exception is java.io.IOException: {  "timestamp" : "2021-03-18T14:18:03.153+00:00",  "status" : 500,  "error" : "Internal Server Error",  "message" : "",  "path" : "/tcs/api/bulk-people/"}

Resolution

• User got reenabled in Keycloak.

Timeline

• 18 Mar 2021: 12:52 Mike Richardson reported an issue with Bulk upload on Teams.

• 18 Mar 2021: 13:47 Got LOG ID of the file from S3 and tried to re-upload, resulting in the same error.

• 18 Mar 2021: 13:52 User reenabled in keycload, bulk upload went through.

• 18 Mar 2021: 13:54 Asked users to try and re-upload.

• 18 Mar 2021: 14:18 Mike Richardson reported an Internal Server Error as Reason for import failure when uploading a file.

Root Cause(s)

• .

Action Items

Lessons Learned

• .