Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Date

Authors

Andy Dingley

Status

Implementing

SummaryUnable to perform any bulk upload operations

Several TIS services, namely Generic Upload and ESR services, could not authenticate with Keycloak. Resulting in outages in those functions.

Impact

Admins not able to upload any bulk import/update spreadsheets, TODO: ESR impact? ESR Bi-directional not able to import/export changes?

Non-technical Description

...

Trigger

Detection

...

Resolution

...

Timeline

BST unless otherwise stated

  • ??:?? - 14:xx - Message to James Harris querying whether it was possible to do bulk uploads, no indication that they were encountering an issue.

  • 09:44 - TIS Admin reported, via TIS Support Channel, that they were getting an “Unknown Server Error” when performing bulk uploads.

  • 10:47 - Users informed on TIS Support Channel that we were aware of a bulk upload issue affecting all users and were investigating.

  • 12:16 - Users informed on TIS Support Channel that we had deployed a fix for bulk upload.

  • TODO: insert ESR timeline entries from Sentry etc.

...

Root Cause(s)

...

Action Items

Action Items

Owner

...

Lessons Learned