The best blog

My docker nginx proxy let's encrypt websites are down

For no apparent reason, my websites stopped working!

If you are using let's encrypt certbot to generate the ssl certificates for all your domains that point to your docker containers, you may be lucky, we have a possible solution.

If you have already looked at your websites/application containers' logs with docker logs -t --tail 10 my-container-name, and verified that there are no application errors, read on for a possible cause: Docker Let's Encrypt Certbot.

Diagnose

  • check whether nginx-proxy is running (if not try starting it)

    docker ps

  • check if there are any errors with your nginx-proxy container

    docker logs nginx-proxy

    • When runing the logs command, do you see something like:

      nginx: [emerg] SSL_CTX_set_cipher_list("ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DH

E-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384") failed (SSL: error:140E6118:SSL routines:ssl_cipher_process_rulestr:invalid command) ```

If you are seeing the output above, you are probably using jwilder/docker-gen:0.7.3, jrcs/letsencrypt-nginx-proxy-companion along with nginx proxy.

Solution

Find the nginx template

Look for the file that you are using template for generating the certificates: /etc/docker-gen/templates/nginx.tmpl. For example mine is ./nginx.tpl. Another way to find the file is to grep -R "POLY1305:DH" <nginx-proxy-docker-compose-dir>. Replace POLY1305:DH with some part of the string that is output in the error logs as explained above (see Diagnose).

Fix arbitrary new lines

The problem is that new line characters are being introduced in the nginx.tpl file within the cipher strings which are supposed to be one-liners. So open your file, mine is:

vim nginx.tpl

And look for every cipher string that has been split with a new line. Do a pattern search on vim, type:

/ssl_ciphers.*[^';]\n

For every match you want to remove the newline character (use shift+jx)

Conclusion

Now that you have fixed the nginx.tpl cipher strings