I’m trying to add two more certificates to my app for IPv4-only and IPv6-only hostnames. Let’s assume I have a good reason to do this because the only control I have
over clients is the name they hit to.
While “v6” one succeed without any fuss, the “v4” one is still waiting for an issue.
I tried deleting it and adding again (hence the difference of 1 hour between them).
% flyctl certs show v6.bdd.fi -a bdd
The certificate for v6.bdd.fi has been issued.
Hostname = v6.bdd.fi
DNS Provider = googledomains
Certificate Authority = Let's Encrypt
Issued = ecdsa,rsa
Added to App = 5 hours ago
Source = fly
% flyctl certs show v4.bdd.fi -a bdd
The certificate for v4.bdd.fi has not been issued yet.
Hostname = v4.bdd.fi
DNS Provider = googledomains
Certificate Authority = Let's Encrypt
Issued =
Added to App = 4 hours ago
Source = fly
You are creating a certificate for v4.bdd.fi
We are using Let's Encrypt for this certificate.
You can configure your DNS for v4.bdd.fi by:
1: Adding an CNAME record to your DNS service which reads:
CNAME v4. bdd.fly.dev
For Let’s Encrypt to issue the cert, I guess you’re using DNS-01 challenge, hence the presence of _acme-challenge CNAME I needed to define under domain apex. So it should work as I already have this—and it did work for v6.bdd.fi.
% dig +noall +answer txt _acme-challenge.bdd.fi
_acme-challenge.bdd.fi. 4 IN CNAME bdd.fi.xdjk.flydns.net.
bdd.fi.xdjk.flydns.net. 44 IN TXT "89IeLuRvCOicqMXEpS4GYOZY5BpDiyD1z2Gb8Cu2RCg"
Due to the single address family needs I cannot defined a CNAME from these subdomains to the <app>.fly.dev name, but below I verify they’re pointing to the respective addresses of the app.
% [[ $(dig +short aaaa bdd.fly.dev) == $(dig +short any v6.bdd.fi) ]] && echo same
same
% [[ $(dig +short a bdd.fly.dev) == $(dig +short any v4.bdd.fi) ]] && echo same
same
There isn’t any logs for me to debug and understand what is not working out.
I’d appreciate any help.
We do ALPN challenges when an AAAA record is present, so that’s why v6.bdd.fi works.
v4.bdd.fi needs its own acme challenge entry, though: _acme-challenge.v4. This is not obvious from flyctl, but if you look at it in fly.io/apps/ you’ll see better instructions.
I was reading RFC 8555 and despite the clear example of ‘www’ there I wrongly assumed equivalence for all the subdomains. Though it’s not. I re-read after your answer assuring me I misunderstood the requirement.
Well, would you look at that. I legit didn’t know about this interface at all. I don’t think I came across it anywhere in the docs. Somehow didn’t notice it when I logged in again to add 2FA. I just saw “Account” link up there.
Wildcard even at one level up is not something I want to trust to third-party for now. Just to make sure I don’t mess this up, I even added issuewild ";" CAA entry to DNS. Otherwise I think you’re right, this is an appropriate choice for most users, also less CT Log noise (2 certs (rsa & ecdsa) per subdomain plus their 2 precerts)
That’s kind of funny, actually, I didn’t realize how hard-to-find we made our UI. The entire onboarding process is CLI first, which is good, but we have neat graphs we should show off!
Resurrecting this for a quick question re: reasoning behind double certs, one with RSA and one with ECDSA key.
@kurt (@thomas reminded to direct this to you)
Do you do this for catering to clients who support TLS 1.2 but for some reason (old browsers, middle boxes, odd IT policies) doesn’t offer any ECDSA ciphersuites?
It’s mostly a legacy choice. We started with RSA, then looked into ECDSA back when we were running a Go based TLS proxy and wanted to make it fast. It was simpler to just add the second cert at the time and we never bothered to change it.
We only bill per hostname, though, you don’t have to double pay for them.
This is probably something we need to revisit. 4 years ago, it made sense to support both (we’ve always prioritized ECDSA). At the time, we also supported TLS v1.1, we only support TLS v1.2 and up now.
Perhaps RSA is not necessary anymore. I’m trying to find information on clients that might only support TLS v1.2 w/ RSA, but it’s proving hard.
According to Qualys, only 1 (very old) Chrome browser on Windows XP used an RSA-only cipher:
Two questions, maybe they’re better in feature request form:
Either drop RSA keys or give an option to add certs with only ECDSA keys.
There’s little to no gain for having RSA backed TLS certificates. As analyzed by @jerome back in Feb 2021, no modern browser lacks ECDSA support. A large majority of IoT devices already support ECDSA for TLS already. A popular TLS implementation of these severely resource and common sense constrained devices, Mbed TLS (née PolarSSL) added support for ECDSA key back in Jan 2014. Anecdotally an ESP01 based, unimaginably bad IoT device I bought back in 2016 ships with software compiled in Nov 2014 and supports ECDSA keys.
If you feel like adding more knobs, maybe instead of straight out retiring RSA support, you may consider offering an option to the user with flyctl certs add --key-type [rsa,ecdsa,both] so determined folks like me can selectively disable RSA keys.
Add SAN (Subject Alternative Name) support to group names into a single certificate
Although a minor issue, this causes quite a bit of certificate transparency log spam. IIUC, certificates are per app anyway; so a permutation of certs across different apps within the same organization is not supported. I can’t think of other good reasons to keep the current behavior of getting a dedicated certificate for every SAN (in fact two per SAN, for two key types). I know Fly charges for certificates beyond the initial 10, but I also know your modus operandi well enough to give you the benefit of doubt that you’ll no such intentions to nickel and dime your users by making both the experience for the users and also the backend for yourselves, unpleasant. I’d gladly pay more than that overage charge of 10¢/mo per cert in terms of “cost of managing certificates and also terminating TLS for you”, but I digress; I’m damn sure your reasoning is not the ability charge a few dollars.
and reduce our system’s complexity.