2018-07-04 Generic Upload failed to boot after an update
Date | 2018-07-04 |
Authors | |
Status | In progress |
Summary | Wednesday morning we were informed that the Generic Upload service was not running |
Impact | No files can be uploaded until this service comes back up |
Root Cause
It is looking like someone used a mysql client on the prod database (like SequelPro).
If you select a table in content mode, and you double click in the empty space it adds a new record with installed_rank 0 and all columns null.
Trigger
Resolution
Removal of the 0 id record from the schema_version table in the database, and then restart the service.
Detection
Detected by Ashley Ransoo & users
Action Items
Action Item | Type | Owner | Issue |
---|---|---|---|
remove write access to prod db for everyone except root and services | prevent | ops | |
Add root password to prod MySQL account for ops use only | prevent | ops | |
Timeline
Supporting Information
e.g. monitoring dashboards
Slack: https://hee-nhs-tis.slack.com/
Jira issues: https://hee-tis.atlassian.net/issues/?filter=14213