...
05:34 - Started recovery of 10.150.0.137/8's docker.
05:41 - Earlier ETLs started failed due to docker issue.
05:47 - Docker recovery worked on ETL box. Need to script now for other machines. Intrepid DR ETL being run at same time then onto Consolidated.
05:50 - Looks like site-dev interacts with the ETL box which could have bumped the version of docker from apt security and killed it. I mean apt security is good but those machines weren't configured correctly in the first place to handle normal operating procedure.
failed: [10.140.0.136]06:04 - Intrepid DR ETL looks like it's been running in dev for a while....
To be discovered.
Detection
...
| Action Item | Type | Owner | Issue |
|---|---|---|---|
| Use docker from their apt rather than ubuntu packaged (docker-ce rather than docker.io) Ansible job. | mitigate/prevent | ||
| Correct setup scripts rather than awful seperated ones | |||
| Better understanding of ETLs needs to be sorted | |||
| Retire https://github.com/Health-Education-England/TIS-DEVOPS/blob/master/ansible/roles/docker-host/tasks/main.yml |
Timeline
Supporting Information
...