2018-07-04 Generic Upload failed to boot after an update

Date2018-07-04
Authors
StatusIn progress
SummaryWednesday morning we were informed that the Generic Upload service was not running
ImpactNo files can be uploaded until this service comes back up

Root Cause

It is looking like someone used a mysql client on the prod database (like SequelPro).

If you select a table in content mode, and you double click in the empty space it adds a new record with installed_rank 0 and all columns null.

Trigger


Resolution

Removal of the 0 id record from the schema_version table in the database, and then restart the service.

Detection

Detected by Ashley Ransoo & users

Action Items

Action ItemTypeOwnerIssue
remove write access to prod db for everyone except root and servicespreventops
Add root password to prod MySQL account for ops use onlypreventops




Timeline

Supporting Information

e.g. monitoring dashboards