| Date |
|
| Authors | Joseph (Pepe) Kelly |
| Status | Complete |
| Summary | |
| Impact | None found. |
Root Cause(s)
Person Elastic Search scheduled job during the night at 1:30am failed to synchronize on different cluster's nodes.
Trigger
The TIS-SYNC application failed with an OutOfMemoryError.
Resolution
Re-ran jobs as on Sync Service.
Detection
Alerting in Slack.
Action Items
| Action Item | Type | Owner | Issue |
|---|---|---|---|
| Restart the jobs | Restore service | Joseph (Pepe) Kelly | TISNEW-3099 - Getting issue details... STATUS |
| Increased JVM Max Heap Size parameter | Prevent recurrence |
Timeline
- 0:29 am TIS-SYNC job failed
- 7.30 am Slack alerting of failed job in #monitoring
- 8.58 am Re-ran jobs
- 1:30 pm change to parameter implemented to reduce the likelihood of a re-occurrence.
0 Comments