2016-12-22 Private Docker Registry refusing access

Date2016-12-22
AuthorsGrante Marshall (Unlicensed) Graham O'Regan (Unlicensed)
StatusComplete
SummaryHEE-TIS Private Docker-Registry was refusing access to its stored repositories
ImpactThis would halt the deployment of updated/new services being made on all platforms, production, staging and development

Root Cause

The docker container was denying all access to its service.  

Trigger

It has been running for months without issue but on the 2016-12-21 a new docker-registry UI docker container was deployed on the same VM and it would appear the two containers running on the same VM may be an issue.

Resolution

Redeploy the docker-registry service using the following procedure:

  1. Login to build server
  2. run the command "ansible-playbook -i /home/jenkins/data/devops/ansible/inventory/dev /home/jenkins/data/devops/ansible/docker-registry.yml"

Detection

While using the docker regisrty UI no repositories were being displayed.  Running curl -u xxxxx:xxxxxx -X GET https://docker-hee.transformcloud.net:5000/v2/hee/revalidation/tags/list?n=100 received curl: (7) Failed to connect to docker-hee.transformcloud.net port 5000: Connection refused

Action Items

Action ItemTypeOwnerIssue




Timeline

  • 13:15 : The docker-registry UI was no longer populating its list = no repositories in its list so started to investigate the issue
  • 13:30 : Redeployed the docker-registry service on the VM
  • 13:31 : docker-registry working and answering requests as designed in both the UI and using curl

Supporting Information