Improving Manual Certbot Domain Validation

In my second post about using Certbot in manual mode, I address some of the 'pain points' from the first post: namely the process of scripting together some of the bits to create/renew a certificate and otherwise requiring fewer individual commands be entered (or remembered).

Key Authentication is Key!

In the original first post of the series, I wrote about setting up key authentication and aliases. Having key authentication is required for further automation with Certbot, and having aliases in place for both connecting to the remote host (e.g. alias server1='ssh -i ~/.ssh/super_private_key user@hostname') and quickly changing paths on the remote host to the dv directory (e.g. alias host-dv='cd /path/to/.well-known/acme-challenge') will turn an laborious process (remembering and typing all the paths) into two commands: server1 followed by host-dv. Done.

First Improvement: Batch Actions

First, for each distinct host (not domain name), I want to create a bash script to generate the certificate(s) and also spit out their key components at the command line (for easy copy/paste). In this example which is named certbot-superdomain.sh, this is a one domain (two variations) certificate for one host, again our fictitious superdomain.net:

#!/bin/bash
certbot certonly --manual --manual-public-ip-logging-ok -d superdomain.net -d www.superdomain.net
read -n1 -r -s -p "Press any key to load certificate...`echo $'\r\n\r\n'`"
more /etc/letsencrypt/live/superdomain.net/cert.pem
read -n1 -r -s -p "Press any key to load private key...`echo $'\r\n\r\n'`"
more /etc/letsencrypt/live/superdomain.net/privkey.pem
read -n1 -r -s -p "Press any key to end certificate renewal process...`echo $'\r\n\r\n'`"

To streamline the previous process, I've used:

• --manual-public-ip-logging-ok. Since I'm fine with that, there's no reason to be prompted for each script, and Certbot has an option to bake that right into the command, so I used it.
• -d superdomain.net -d www.superdomain.net. This allows me to bolt two names into the certificate, in this case both forms with and without the www.
• "wait" prompts between key steps.
• Two 'more' outputs for the certificate and key files.

To invoke this, I simply run sudo bash ./certbot-superdomain.sh. I have also created a Gist of the aforementioned.

Domain Validation Files

As before, the domain must be validated via file, so I am presented with the prompts ala:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Create a file containing just this data:

AkCnpli2xRSUw2vPgR0A8gLUCZtvhyEbIo1aq8_oWYA.QwR5fxV9v8X8jg2GoYFetupYACPp-L9kV8GdQYrS7MI

And make it available on your web server at this URL:

http://superdomain.net/.well-known/acme-challenge/AkCnpli2xRSUw2vPgR0A8gLUCZtvhyEbIo1aq8_oWYA

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Press Enter to Continue

Since I'm running this via WSL in the VS Code terminal, I can use the "Split Terminal" function to get two side-by-side terminals:

The left-hand side is the Certbot process (local), and the right-hand side is the remote server at the host-dv alias.

Some Copy & Paste Action

With the above information, I can literally copy/paste the DV information to the remote server with two commands:

• vi AkCnpli2xRSUw2vPgR0A8gLUCZtvhyEbIo1aq8_oWYA
• In Vim, use i to enter insert mode, paste the data value (AkCnpli2xRSUw2vPgR0A8gLUCZtvhyEbIo1aq8_oWYA.QwR5fxV9v8X8jg2GoYFetupYACPp-L9kV8GdQYrS7MI), escape to exit, and shift-ZZ to save and exit.

Now, as a Vi/Vim user, these commands really flow for me. They might not be your jam, but there are plenty of ways to create and populate a file on the command line (more about what I did for full automation in the next post). The important improvement part is in having the split terminal, since it's so easy to move between them. Do what works for you.

When you're done adding files, the validation process will work just as it usually does in manual mode, and should issue a certificate if all went well.

The Certificate Components

I added the two 'more' outputs to spit out the full cert and private key data at the prompt. I can easily then copy/paste these into the cPanel fields, and since these are scripted together I always know I'm using the proper output files.

What About Multiple Domains per Host?

I have several hosts with multiple domains (variations of the same name, vanity domains, etc.). The original certbot-superdomain.sh can be easily modified to include as many distinct domains as necessary. Each unique "domain" itself, though, will need its own certificate, so while you can add multiple subdomains (e.g. with and without www) to one command, distinct names for the same host will need to have their own certificates. For example, this is a simple way for two variations of superdomain.net to be bolted into the same script, this time called certbot-multi-superdomain.sh:

#!/bin/bash
certbot certonly --manual --manual-public-ip-logging-ok -d superdomain.net -d www.superdomain.net
read -n1 -r -s -p "Press any key to load certificate...`echo $'\r\n\r\n'`"
more /etc/letsencrypt/live/superdomain.net/cert.pem
read -n1 -r -s -p "Press any key to load private key...`echo $'\r\n\r\n'`"
more /etc/letsencrypt/live/superdomain.net/privkey.pem
read -n1 -r -s -p "Press any key to begin renewal process for superdomain.com...`echo $'\r\n\r\n'`"
certbot certonly --manual --manual-public-ip-logging-ok -d superdomain.com -d www.superdomain.com
read -n1 -r -s -p "Press any key to load certificate...`echo $'\r\n\r\n'`"
more /etc/letsencrypt/live/superdomain.com/cert.pem
read -n1 -r -s -p "Press any key to load private key...`echo $'\r\n\r\n'`"
more /etc/letsencrypt/live/superdomain.com/privkey.pem
read -n1 -r -s -p "Press any key to end certificate renewal process...`echo $'\r\n\r\n'`"

In this example, two domain names are bolted together for certificate issue (superdomain.net and superdomain.com). This can be a handy way to "step through" all variations for a given host at the same time. Each individual step is the same as for a single domain. I've created a Gist for certbot-multi-superdomain.sh as a quickstart.

What's Next?

While this scripted together business was a huge improvement over what I'd been doing initially, it still required a lot of domain validation file fiddling. For just a few certificates this wasn't too bad, but I'd expanded to using this process for about 18 distinct domains at the time, which still caused for some pain...even though the whole process really takes about 15 minutes per quarter.

In the next/final installment of the series, I will go over what I did to really bolt it all together, to the point where I have removed the need to do any split terminal or remote host work (at the command line, that is) for domain validation. With the magic of key authentication and remote commands, the hook functionality in Certbot makes it possible to automate much more!