Root Cause

The LetsEncrypt certificates had expired because the renewal hadn't been automated.

Trigger

The expiry date of the certs was reached.

Resolution

The certs were manually renewed using the following process;

1. Login to the VM

2. run the following command: 

   $ sudo letsencrypt renew
   Processing /etc/letsencrypt/renewal/build-hee.transformcloud.net.conf
   2016-12-22 10:24:10,890:WARNING:letsencrypt.cli:Attempting to renew cert from /etc/letsencrypt/renewal/build-hee.transformcloud.net.conf produced an unexpected error: Failed authorization procedure.dev-api.transformcloud.net (tls-sni-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: DNS problem: NXDOMAIN looking up A for dev-api.transformcloud.net. Skipping.

3. Check that the new certs have been created check the following directory for new entries; 

   $ sudo ls -la /etc/letsencrypt/archive/

4. Restart the webserver 

   $ sudo systemctl restart apache2.service

Detection

The team recieved warnings when trying to view Jenkins.

Action Items

Timeline

• 10:30 Grante Marshall (Unlicensed) noticed that Jekins cert weren't working
• 10:39 Grante Marshall (Unlicensed) ran the update command but hit issues with the fact that we had certs for an invalid domain as we are no longer using dev-api.transformcloud.net

Supporting Information

None