Port 53 UDP not working

1

Hi! I’m working on migrating backend infrastructure to fly and am running into some odd behavior. I am attempting to serve a custom DNS server via port 53 publicly and port 5300 in the docker container. I’ve set up my configuration and have found that I can successfully serve TCP requests, but not UDP requests via port 53 however if I change the external service port to be 5300, I am able to issue requests (but have to specify that 5300 port specifically). Is this a known issue or are there any workarounds?

I also was able to replicate this behavior using the example Go server given in the documentation (udp-echo-/fly.toml at main · fly-apps/udp-echo- · GitHub)

fly.toml:

app = "<my-app-name>"

kill_signal = "SIGINT"
kill_timeout = 5
processes = []

[env]
  PORT = "5300"

[build]
  dockerfile = "./dns.dockerfile"

[experimental]
  allowed_public_ports = []
  auto_rollback = true

[[services]]
  internal_port = 5300
  protocol = "udp"

  [[services.ports]]
    # This changed to 5300 will let me issue requests
    port = "53"

[[services]]
  internal_port = 5300
  protocol = "tcp"

  [[services.ports]]
    port = "53"
2

I made the same observation with the DNS port 53/UDP example, and saw it again with my TFTP port 69/UDP example.

So far, I could not figure out the reason or a countermeasure why UDP fails to work when external & internal ports differ and port address translation becomes necessary. From the logs, it looks as if requests make it to the server inside the microVM OK. But its answers do not make it back to the client somehow.

Eventually, is it connected to the speciality with UDP on Fly.io, where the server must bind to the address fly-global-services. Or, does eBPF fail at the reverse port address translation of outgoing reply packets “somehow”?

3

Not sure if this is the reason, but do note that IPv6 over UDP isn’t supported, yet.

4

While UDP over IPv6 is not yet supported AFAIK, we observe the problem with UDP over IPv4-only servers.

5

We’re looking into this, here’s a few things you can try:

  • make sure you’re binding to 0.0.0.0 for your UDP service
  • try binding directly to port :53

I believe your config is correct, but something else is breaking the functionality.

6

Hey Jerome, thanks for the pointers. To clarify, you’re saying I should be binding to 0.0.0.0 for UDP services in lieu of the fly-global-services that’s mentioned in the docs?

1 Like
7

Clarification would be welcome because Running Fly.io Apps On UDP and TCP states:
“The UDP side of your application needs to bind to the special fly-global-services address. But the TCP side of your application can’t; it should bind to 0.0.0.0 .”
That’s what I have implemented and tested so far.