Certificate expired and not renewing

[bings]

Our certificate expired today, and it’s not been renewned. The domain is verified, so I’m not sure why not?

[bglw]

I think this is a case of the ACME DNS record conflicting with Cloudflare’s Universal SSL. Since these records can conflict, our backend might get the expected value, while Let’s Encrypt does not. This is a common pain point, and in fact is one reason why we just released new certificate workflows to help improve this in Easier certs behind Cloudflare, BYO certs, and other goodies.

If you configure the Ownership TXT Record listed on your dashboard there, this will successfully issue the certificate in most cases. If that still fails, using a Cloudflare origin cert is the most bulletproof option.

[bglw]

Also of note: Removing the _acme-challenge CNAME from your DNS might help after adding the ownership TXT record, just to ensure it doesn’t keep retrying the DNS challenge path.

[bings]

So I imported the cloudflare origin certificate, added the _fly-ownership and removed the _acme-challenge. The fallback fly certificate was re-issued after a few minutes (however I guess that didn’t matter anymore), but when I went to my domain I would see:

So I’ve then removed the origin certificate incase it’s somehow that, but the issue still persisted.

There was no errors (or logs, at all) on the fly machine. I tried restarting the fly machine (which was working fine prior to today, and had no recent deployments), but still no dice.

In a bit of a hail mary, I turned off the proxy/‘Orange cloud’ on the A, AAAA, and CNAME (www) records (which had been enabled for the past 8 months).

After 5 minutes, it suddenly started working again. No idea if that had any impact, but if it’s working… it’s working…

[bglw]

Hmm. Turning off orange cloud would definitely have the biggest impact, as Fly.io can terminate the TLS directly. Off the top of my head I’m not familiar with how one could cause the “Host” error between Cloudflare and our end. Any logs regarding that error would have to be sourced from the Cloudflare end, in all likelihood. I’ll keep an eye out though. I would be surprised if turning orange cloud back on caused any issue, but I also totally understand not wanting to touch it once it is working.