Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Date

Authors

Rob Pink Joseph (Pepe) Kelly Yafang Deng

Status

Documenting

Summary

The absence of notifications was clocked and on investigation it was found that none had been sent from 10 August

Impact

Jira Legacy
serverSystem Jira
serverId4c843cd5-e5a9-329d-ae88-66091fcfe3c7
keyTIS21-6411

...

Non-technical Description

Trigger

Detection

Resolution

...

Notifications are usually sent to ESR every day, describing changes and updates to the “now & next” people in positions.

The service did not generate the files for 11th - ?? .

...

Trigger

...

Detection

We noticed a lack of “Confirmation” messages from ESR

...

Resolution

...

Timeline

  • Shovel setup but was persistent rather than temporary.

  • notifications not exported since

  • issue noticed

  • 11:30 resized cluster, and it took 12mins

  • 11:50 deleted the shovel which was moving every error to esr.dlq.all.shovel.31072024.

5 Whys (or other analysis of Root Cause)

...