...
| Date |
|
| Authors | Joseph (Pepe) Kelly |
| Status | Complete |
| Summary | |
| Impact | None found. |
| Table of Contents |
|---|
Root Cause(s)
Person Elastic Search scheduled job during the night at 1:30am failed to synchronize on different cluster's nodes.
Trigger
The TIS-SYNC application failed with an OutOfMemoryError.
Resolution
Re-ran jobs as on Sync Service.
Detection
Alerting in Slack.
Action Items
| Action Item | Type | Owner | Issue | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Restart the jobs | Restore service | Joseph (Pepe) Kelly |
| ||||||||||
| Increased JVM Max Heap Size parameter | Prevent recurrence |
Timeline
- 0:29 am TIS-SYNC job failed
- 7.30 am Slack alerting of failed job in #monitoring
- 8.58 am Re-ran jobs
- 1:30 pm change to parameter implemented to reduce the likelihood of a re-occurrence.