SSL not working when using custom domain

At some point semi-recently SSL using custom domains stopped working for one of my apps. It’s a low-traffic app which I only access every couple of weeks, so it took a while for me to notice it was down.

The * URL ( works as expected, and the app displays.
The custom domain ( however, does NOT, because there’s some issue with SSL/TLS. All the correct DNS records are in place.

To clarify - this is something that used to work, and broke at some point without me doing any changes. Based on my understanding the whole SSL-termination is done by, so as long as my app is up and running I’m expecting SSL for custom domains to “just work” (as long as DNS records and certs are in order).

Here’s some diagnostics info:

# Firefox error message:
An error occurred during a connection to PR_END_OF_FILE_ERROR

# Diagnosing it a bit more with openssl directly:
$ openssl s_client -connect                                                                                                                         
4047DD54CB7F0000:error:0A000126:SSL routines:ssl3_read_n:unexpected eof while reading:ssl/record/rec_layer_s3.c:321:

# Output of `fly certs check':
$ fly certs check '*'                                                                                                                                     
The certificate for * has been issued.
Hostname                  = *
DNS Provider              = aws
Certificate Authority     = Let's Encrypt
Issued                    = rsa,ecdsa
Added to App              = 3 months ago
Source                    = fly

Please let me know if there’s any other piece of diagnostics info that might be useful. I’m afraid I’m a bit stuck not knowing what to check next.

You’ll need an extra certificate for the root hostname This is a common issue we should fix by adding the root hostname as an additional servername on the certificate.

That’s spot on @jerome - thanks a bunch!
As soon as I did fly certs add things started working.

It would be great to do just like you suggested - to add the apex domain as an additional server name on the certificate. Or perhaps even modify the UI / CLI a bit to make it a bit more obvious that things might not be properly set up. I guess I got a bit tricked by all the green checkmarks, and didn’t think properly about what might be wrong :slight_smile:

For reference for other readers of this thread:

To have SSL properly set up for the apex domain (e.g. and any of it’s subdomains (e.g. the output of your fly certs list should look something like this:

$ fly certs list                                                                                                                                                

Host Name           Added                Status         5 minutes ago        Ready
*       3 months ago         Ready

I do have a similar issue.

fly certs list
Host Name                 Added                Status          3 months ago         Awaiting configuration

it was working fine until today, haven’t changed anything in months, last deployment was yesterday. what’s going on?

EDIT: fixed by removing and adding the certificate again. I’ve expected certificate to regenerate automatically

If you go to “Certificates” in the Fly control panel you can click “Check again”. Does that give any insights?

I believe you need to have all the DNS records in place for things to work properly. Do you still have the A record and the CNAME (ACME Challenge) record set up? I’m asking because previously I’ve done the mistake of “cleaning up” (deleting) the ACME Challenge record, because I thought it wasn’t needed after initial setup. I’m thinking it’s probably needed when renewing the certificate as well.

EDIT: I just saw that you removed and added the certificate again and that fixed things. Since it stopped working after about 3 months it very much seems like the auto-renew of the certificate didn’t work for whatever reason…just like you said.

Didn’t have time to look at the possible error output for that check.

It was added 3 months ago, and usually we renew every 2 months.

@mkozak it’s possible the hostname didn’t pass our own verification checks and we kept rescheduling the verification further away in time. I can’t tell anymore because we don’t store each verification attempt’s output (that would be a lot of data).

1 Like

I’ve found the issue - I must have accidentally deleted the verification cname record in the past 3 months. That’s why it was working fine during this time, but when it tried to renew couldn’t verify my ownership of the domain (A and AAA records were intact)


@mkozak Been there, done that. :slight_smile: Great that you found the issue!

@jerome Thanks for the info on the renew cycle. Great to know when troubleshooting these kind of things.