2019-09-05 TIS Service Unavailable
- Liban Hirey (Unlicensed)
- Oladimeji Onalaja (Unlicensed)
| Date |
|
| Authors | Oladimeji Onalaja (Unlicensed) |
| Status | Complete |
| Summary | |
| Impact | Service Unavailable errors on TIS |
Root Cause(s)
The implementation of the sync service required the api-gateway playbook run and this inadvertently changed the versions of apache and modsecurity
As a direct consequence, TIS users were getting intermittent 503 "Service Unavailable" errors.
Trigger
The number of valid state cookies would exceed the limit which was set to 3 cookies at the time.
Resolution
Updated Open ID Connect from version 2.3.9-1 to 2.3.11-1 as this version includes an option to delete the oldest state cookie once the maximum number of cookies has been reached.
Maximum number of cookies set to 1.
Detection
Alerted by users.
Action Items
| Action Item | Type | Owner | Issue |
|---|---|---|---|
| ?? | Restore service | TISNEW-3270 - Getting issue details... STATUS |
Timeline
- August 29, 2019 - TIS Service Unavailable error appears
- September 2, 2019 - Root cause of errors located
- September 3. 2019 - Fix implemented
Slack: https://hee-nhs-tis.slack.com/
Jira issues: https://hee-tis.atlassian.net/issues/?filter=14213