Certificate stuck at Awaiting certificates

Hi!

I’m using Cloudflare for my DNS, which from what I’ve gathered might cause issues.

I’ve managed to disable SSL/TLS on the Cloudflare site, and for the DNS I’ve disabled the proxying so the IPs should correctly be showing up (and not showing Cloudflare IPs).

The www.zigzagger.io domain certificate has correctly been issued, but the root one is stuck at Awaiting Certificates

I’ve also added the acme challenge DNS record, which seems to be showing up correctly.

I’ve tried setting up the root domain with the CNAME flattening, and the plain IP addresses, but the certificate still doesn’t seem to be issued.

I’ve tried removing and re-adding the certificate a few times, but still not getting anything.

I’m guessing I’ve still misconfigured something, but I’m out of ideas at the moment!

Any ideas?

Thanks,
Phil

hey there-- you might have already noticed this, but we’re currently investigating issues with our private DNS servers. If you’re using a DNS-01 challenge (and it sounds like you are), then this is most likely what’s holding you up.

You also can follow our progress on our status page as we work to get everything back in order as soon as we can.

Ah, hadn’t noticed, so thanks for the heads up!

I’ve been having the issue for the past 2 days, and this looks like it might be a more recent issue?

Just to jump in, this may be related to the DNS issue but if you say it was happening prior to that, that may equally be a coincidence.

You mention “the root one”. Do you mean the apex domain? If so, I recall a mention you needed both A and AAAA records to validate that. I’m not sure if that’s still the case but could be worth a try. Can’t hurt. From Fly:

1 Like

Yup, it’s definitely still the case that you need a A and AAAA record to get a cert for your apex domain by itself. For root domains. we use TLS-ALPN-01. A wildcard cert for an apex domain would use a DNS-01 challenge, though.

We have a fairly in-depth blog post covering our ACME setup and some of the design decisions that went into it, which you might find useful and/or interesting, too.

3 Likes

Hey Greg,

Thanks for jumping in! Yeah, by root I meant apex, just tired and my brain is not firing on all cylinders!

On the apex domain I’ve tried setting the A and AAAA, and the CNAME flattening (which appears to have the exact same result), and it still wasn’t being issued.

I’ll wait and see what happens with the ongoing issue, maybe it’ll just get solved with this!

Thanks

1 Like

Looks like the issue has been resolved, but the certificate is still stuck at Awaiting certificates :confused:

Hmm … :frowning:

Assuming both A and AAAA DNS records remain grey-cloud (non-proxied) in Cloudflare (else they definitely won’t work) you could maybe try requesting a certificate again, to give it a kick?

Else this is one for Fly to pick up and debug at their end. They may need the output of flyctl certs show your-domain.com. I don’t think that reveals anything sensitive, just DNS records.

FWIW I’ve got the same issue - www version got created instantly and serves the website, but the naked domain is stuck on creating the certificate.

Also using Cloudflare and turned off proxy from everything for now - set up A and AAAA records for both apex and www domains

Once this is resolved / we figure out what we’re doing wrong, can we enable the Cloudflare proxy again?

Ah.

Yep, again one for Fly to investigate. Likely be tomorrow now.

I suspect re-enabling the Cloudflare proxy would cause any subsequent re-validation to fail. In X weeks/months, whenever it checks again. Since then it would return a Cloudflare IP. Not a Fly one. :thinking: Now I would think the grey-cloud/non-proxied acme challenge would be sufficient to validate the www record (and so let you orange-cloud/proxy the actual www record). But I don’t think the acme challenge would be sufficient for the apex record. I’ve seen multiple posts where the apex has to be A/AAAA IP … and so you would then run into that problem of it returning the proxy’s A/AAAA IP.

2 Likes

I was digging through Cloudflare to see if there are any more things I can disable that might be interfering here. Found a button that says “Disable Cloudflare on site”, which disabled all Cloudflare features and just continues to resolve the DNS records.

Removed and re-created the certificate, but not having any luck sadly. I’m guessing disabling all of Cloudflare other than DNS does eliminate Cloudflare as the issue here (if I understand that feature correctly), so maybe it is indeed an issue within Fly.

Any ideas @eli?

Thanks!

So I tried this out for myself with an identical record config with Cloudflare on a test domain, which succeeded. Digging into things on our end, it looks like the _acme-challenge record (the acme CNAME for the apex domain) might be getting in your way, though. Would you try removing it, to see if that helps?

1 Like

I’ll try it, thanks! When I remove the CNAME, should I remove and re-add the domain in fly?

Thanks

It shouldn’t be necessary to re-add the CNAME for the _acme-challenge records unless you need DNS-01 verification (which is why the record is pointed to the nameservers we use for that).
You’d need DNS-01 verification if you were trying to create a wildcard certificate.

Hopefully removing the acme-challenge records will sort things out!

Thanks @eli

I’ve removed the acme challenge record, and deleted and re-added the certificate in fly.

No luck in 15 minutes so far (I remember the www certificate was issued in roughly 1 minute)

FWIW it worked for me after about 15 minutes!

Sorry to hijack the thread but since we’re both dealing with Cloudflare I hope you don’t mind @manavo - just wanted to recap on the proxying through CF aspect - it sounds like to be able to use the benefits of CloudFlare not worry about the expiry of the certificates we’d need to just use a CNAME for the apex domain pointing to the .fly.dev URL? Which adds an extra network hop but keeps the benefits?

And this wouldn’t be possible if we wanted to use a wildcard domain at all? (keeping the benefits of CF proxy and not worry about cert renewal)

1 Like

@manavo, it looks like both zigzagger and www.zigzagger.io are now serving certs, so hopefully this means that our certs came through!

@tomasztomczyk: Yup, if you needed a wildcard domain, you’d need a DNS-01 challenge. Let’s Encrypt (our CA) won’t issue them otherwise.

Keeping Cloudflare’s proxied records on will prevent our TLS-ALPN-01 challenge from being completed successfully. which is what you’d be using without an _acme-challenge record. I’m not completely sure why these records seemed to be causing issues for your cases-- when I tried setting up records identical to yours (but with a different domain), it succeeded within a couple of minutes.

If you need Cloudflare proxying, It’s definitely worth a shot to try re-adding these records at some point, perhaps creating them with a TTL of 2-3 min and waiting to make sure they’ve updated globally with a tool like dnschecker.org or similar.

That’s great, thanks again for your help

I’m having the exact same issue. Moving a domain (tiffr.com, both apex and wildcard) that has DNS records with Cloudflare to point at Fly. All Cloudflare proxying is currently turned off. Domain is verified, but cert is not issued. Removed and re-added the certificate request once to see if that would kickstart it, but hasn’t helped. Any advice?

Also, as a follow up question: will there be the option to force using the DNS-01 challenge, even for non-wildcard domains (or is this already the case)? That would be helpful for avoiding the need to turn off proxying of DNS records other than the _acme-challenge CNAME at Cloudflare for renewals.