Over the last few weeks, we have been tidying up some edges of the certificate issuance flow:
Improved *.fly.dev setup
After you create an app, one of the jobs we need to do is start routing <your-app-name>.fly.dev to it. A rather important part of serving that traffic is attaching a valid certificate, and until recently this was sometimes causing a delay until your app was truly available.
We’ve always served these using our wildcard certificate for *.fly.dev, but we required a step where our backend would alias that certificate to your app in Vault (our certificate store). As with many things that traverse our web backend, if queues are backed up or talking to Vault fails, this alias isn’t ready by the time you click your new app link.
There wasn’t a great reason for things to work this way, it’s just the way it was. So now, there’s a new way. Fly Proxy has taken on the job, and can serve the *.fly.dev certificate directly for your apps. It’s not a very flashy improvement, but since this certificate is all-but-guaranteed to be cached at our edge, it means your app domains are available faster.
Improved certificate distribution
One of the points mentioned above — “or talking to Vault fails” — also affected adding custom domains. Sometimes services get busy and your certificate would be saved, but adding it to our Vault cluster would fail. It would still eventually get there, but probably not until you kicked it by running fly certs check or clicking “Check again” in the dashboard. We now retry this job a whole lot better, and certificates are more reliably available after generation.
If things do go wrong… improved error messaging
Adding a hostname and asking us to generate a certificate can fail in at least 13 different ways. Usually your DNS records aren’t quite right, and between our backend and Let’s Encrypt, someone is going to complain and refuse to create a certificate in your name.
We had decent error logging on our end to help customers debug, but if you were checking the dashboard or running fly certs check you’d only get a generic error, or a red dot showing that something was wrong.
Now, we do our best to give you something helpful to work with, both in the CLI and in the web dashboard. If you don’t have an IPv6 address allocated, or do but haven’t set your AAAA records, or have a CAA record that is wrong, or a number of other things: we’ll pop up a useful message saying so, and you can iterate on your setup until your certificate is ready to roll.
Hopefully, in sum, these changes make adding certificates more reliable and more helpful. Let us know if you’re trying to add a certificate and get stuck! There’s likely lots more we can test for and catch in our error reporting going forward.