We’ve shipped some big improvements to how certificates work on Fly.io. There’s a new way to verify domain ownership that sidesteps IPv6 issues behind CDNs like Cloudflare, alongside the ability to import your own certificates. Plus, certificate changes now propagate to all edges immediately, and we have new endpoints for managing certificates through the Machines API.
The _fly-ownership TXT record
When you add a certificate to your app, we need to verify two things: that Let’s Encrypt can issue a cert for your domain, and that your domain specifically points to your app (not just to Fly.io in general). The latter is important: IP addresses and app names can be recycled, so without strict verification, someone could pick up a released IP and issue a certificate for a domain that’s still pointing at it. This is subdomain takeover, and it’s the reason we’ve historically required an IPv6/AAAA record [1].
The problem is that CDNs like Cloudflare obscure your destination IPs by design. We added HTTP validation a while back to work around this — it punches through the CDN and verifies that the ACME challenge arrives over IPv6. This works, but it requires proxying strictly to an AAAA record, which isn’t ideal.
That’s no longer the recommendation, as we now support a _fly-ownership TXT record to prove you control your domain. If this record is set, we forgo any IPv6 requirements, which means you can configure the proxied DNS records however you need.
If you run fly certs setup, or check in your app dashboard, you’ll see the new ownership record listed. Any certificates already issued don’t need to change, and the previous AAAA-only setup will continue to renew.
Sometimes, though, this still doesn’t quite work out. Hence:
Custom certificate imports
You might find that even the ownership TXT record isn’t enough. For example, with certain strict Cloudflare configurations, Let’s Encrypt isn’t able to get through to Fly.io with the /.well-known/* HTTP-01 challenge. Or, if you’re setting up a wildcard certificate, you might find that the _acme-challenge record gets clobbered by Cloudflare’s Universal SSL, breaking the DNS-01 challenge. With the TLS-ALPN challenge unavailable due to Cloudflare being in the way, we simply run out of ACME challenge types to use.
For these reasons, you can now import your own certificate. In the dashboard, there’s a new import option on the certificates page, or you can run fly certs import via the CLI.
This is particularly useful with Cloudflare origin CA. Cloudflare lets you generate a certificate that they trust, valid for up to 15 years. Attach that to your Fly App, and the Cloudflare ↔ Fly.io connection is encrypted without needing an ACME cert at all. And then just set a reminder to do that again in 2041.
You do still need to prove domain ownership via an AAAA record, or the _fly-ownership TXT record, before a custom certificate will become active. The CLI and dashboard will walk you through this.
Custom certificates and ACME certificates can coexist on the same hostname. Your custom cert takes priority, and the ACME cert acts as a fallback.
See our documentation for instructions on uploading a custom cert, and our specific documentation for using a Cloudflare origin certificate.
Certificate changes now propagate to all edges
Previously, when a certificate was reissued or replaced, the Fly Proxy would keep serving a cached version until it expired. Different edges could serve different versions of a certificate for the same hostname, which was at best confusing, and at worst caused real issues when deleted certificates were unexpectedly served.
Certificate changes now actively invalidate the proxy cache. When a cert is issued or deleted, the change propagates to all edges ~immediately.
New Machines API endpoints
All certificate operations now have Machines API endpoints, which is what flyctl and the dashboard use under the hood. If you’re automating certificate management, these are the endpoints to use going forward. The existing GraphQL API still works and isn’t going anywhere any time soon.
IPv6 is such a plentiful resource that we won’t need to recycle any addresses for a few quadrillion years, which means an IPv6 address uniquely associates a domain to a single Fly App. ↩︎