fly.dev intermittent DNS issues?

I notice now that the dashboard says the app has no IP address. Was it supposed to create one automatically?

EDIT: indeed, assigning an IP address solved the missing DNS. Why didn’t the documentation instruct me to assign an IP address?

Even after setting an IP address, I somehow was not able to access my running instance of pentacent/keila:latest and when trying to fix it, flyctl asked me to put in my credit card information, which I have no desire to do until I can validate that my use case can work. By for now fly.io

@beeb I don’t understand most of what you just described. I don’t believe the issues you ran into were related to DNS, though.

You should be able to access an app publicly without having to update a credit card form. If you’re up for posting details about what you ran into, I’m sure we can help.

Hey kurt, you are right that it was not in the end a DNS issue for me. It was simply the system not assigning an IP for my app by default, and the instructions in the docs (I followed the dockerfile deployment page) not mentioning this. Even after solving this, my app would somehow still not work, and I wanted to try to redeploy with different environment variables, and this is when it asked me to fill a credit card. I switched to railway.app which worked on the first try.

1 Like

I think the issue is: we don’t allow deploys of images from Docker Hub without a credit card. It sounds like that might’ve worked once for you, which is interesting. In general, you need a credit card on file to deploy from Docker Hub.

Railway is great. I’m glad it worked for you.

1 Like

@beeb, fly.io is awesome – you should give it a longer try. This is an unusual outage issue for them. I’m a little surprised they haven’t disabled their DNS servers which seem to be causing intermittent problems for days – but hopefully they can fix it soon.


I’m still seeing the issue this morning, for example three consecutive runs checking Google Public DNS 8.8.8.8 fails for fly.dev (and also our subdomain):

$ host fly.dev 8.8.8.8
Using domain server:
Name: 8.8.8.8
Address: 8.8.8.8#53
Aliases: 

fly.dev has address 77.83.140.34
fly.dev has IPv6 address 2a09:8280:1:a270:b8f5:b9f7:8891:f3

$ host fly.dev 8.8.8.8
Using domain server:
Name: 8.8.8.8
Address: 8.8.8.8#53
Aliases: 

fly.dev has address 77.83.140.34
Host fly.dev not found: 3(NXDOMAIN)
Host fly.dev not found: 3(NXDOMAIN)

$ host fly.dev 8.8.8.8
Using domain server:
Name: 8.8.8.8
Address: 8.8.8.8#53
Aliases: 

Host fly.dev not found: 3(NXDOMAIN)

The answer to this StackOverflow post might be helpful:

The problem is that the authoritative nameservers for cdn77.org fail to properly handle ECS(EDNS Client-Subnet) options when they contain an IPv6 client subnet, although they handle IPv4 client subnets just fine.

Could this be the root cause – according to this EDNS Compliance Tester, the DNSimple nameservers pass but the FlyDNS nameservers might not be in compliance?

There’s a troubleshooting Google Public DNS webpage, but I don’t notice anything else obvious.

We’re working on it. This is not obviously a problem with our nameservers. It also takes like 48 hours to get nameservers to propagate, so there’s no quick fix.

We’re trying to ask Google. They happily return a result from each or our name servers in our tests. We never return an nxdomain for an app, so it’s really unclear why that’s happening.

The current theory is that their cache is poisoned from a previous nxdomain error.

Is it also possible nameserver requirements is a problem?

Consistency between authoritative name servers

The data served by the authoritative name servers for the designated zone must be consistent.

All authoritative name servers must serve the same NS record set for the designated domain.

All authoritative name servers must serve the same SOA record for the designated domain.

$ dig @ns1.flydns.net fly.dev SOA
fly.dev.		3600	IN	SOA	ns1.flydns.net. ops.fly.io. 1676570065 86400 7200 604800 300

$ dig @ns1.dnsimple.com fly.dev SOA
fly.dev.		3600	IN	SOA	ns1.dnsimple.com. admin.dnsimple.com. 1552047096 86400 7200 604800 300

@kurt ah yes that explains the credit card, I also switched from a 1-line dockerfile (FROM …) to a line in the config file since my dockerfile was not doing much. I didn’t know one needed a CC and the other didn’t. Anyway, I might give it another go when I get a chance.

@mrjbq7 are you still having issues?

We are removing the DNSimple nameservers. These might’ve indeed been causing issues.

Hopefully this resolves the last problems.

Yes, it appears as of this post that resolving via 8.8.8.8 sometimes does not return an AAAA record and sometimes does not return A or AAAA records. When resolving via 2001:4860:4860::8888, it is also still failing intermittently:

# loop running host so we can detect flaky responses
$ while true; do host fly.dev 2001:4860:4860::8888; sleep 0.5; done; 
Using domain server:
Name: 2001:4860:4860::8888
Address: 2001:4860:4860::8888#53
Aliases: 

Host fly.dev not found: 3(NXDOMAIN)
Using domain server:
Name: 2001:4860:4860::8888
Address: 2001:4860:4860::8888#53
Aliases: 

Host fly.dev not found: 3(NXDOMAIN)
Using domain server:
Name: 2001:4860:4860::8888
Address: 2001:4860:4860::8888#53
Aliases: 

fly.dev has address 77.83.140.34
fly.dev has IPv6 address 2a09:8280:1:a270:b8f5:b9f7:8891:f3
Using domain server:
Name: 2001:4860:4860::8888
Address: 2001:4860:4860::8888#53
Aliases: 

Host fly.dev not found: 3(NXDOMAIN)
Using domain server:
Name: 2001:4860:4860::8888
Address: 2001:4860:4860::8888#53
Aliases: 

fly.dev has address 77.83.140.34
Host fly.dev not found: 3(NXDOMAIN)
Host fly.dev not found: 3(NXDOMAIN)

We did the workaround suggested above where instead of just a CNAME that resolves using the fly.dev nameservers, we put the IPs in our domain, so that’s working to avoid the problem.

I’m sorry, I tried to reply right away to your question but my replies are limited/delayed as a newer user of this forum, even though I’ve had a fly account for awhile…

1 Like

Why is flydns returning non-authoritative answers?

$ nslookup fly.dev ns2.flydns.net
Server:		ns2.flydns.net
Address:	137.66.40.8#53

Non-authoritative answer:
Name:	fly.dev
Address: 77.83.140.34

$ nslookup fly.dev ns2.dnsimple.com
Server:		ns2.dnsimple.com
Address:	2400:cb00:2049:1::a29f:1904#53

Name:	fly.dev
Address: 77.83.140.34
1 Like

We are reverting this change for now, and have opened an incident on our status page: Fly.io Status - DNS Resolution for fly.dev occasionally failing

This is an old topic but I’m unable to resolve my app’s dns as well, not even from ns1.flydns.net. just deployed.