Inbound UDP not working

user115 · February 15, 2022, 3:33am

Hi, I’m experimenting with fly.io as an authoritative DNS platform. I’m running nsd with a standalone configuration. After my server starts, it’s reachable over TCP (e.g. with dig +tcp @myserver) but not UDP. If I connect to the console with SSH and use dig against localhost and the IP everything works as expected.

My fly.toml file looks like this:

app = "mydnstestapp"

kill_signal = "SIGINT"
kill_timeout = 5

[[services]]
  internal_port = 53
  protocol = "udp"

  [[services.ports]]
    port = 53

[[services]]
  internal_port = 53
  protocol = "tcp"

  [[services.ports]]
    port = 53

Any pointers appreciated

jsierles · February 15, 2022, 11:33am

Can you paste your server config? You may need to explicitly bind to :: or to fly-global-services.

user115 · February 15, 2022, 1:52pm

Hi! Thanks so much for the reply. I have the ip-address sections commented out, the defaults should be to bind to all local interfaces. It looks like this:

# uncomment to specify specific interfaces to bind (default are the
	# wildcard interfaces 0.0.0.0 and ::0).
	# For servers with multiple IP addresses, list them one by one,
	# or the source address of replies could be wrong.
	# Use ip-transparent to be able to list addresses that turn on later.
	# ip-address: 1.2.3.4
	# ip-address: 1.2.3.4@5678
	# ip-address: 12fe::8ef0

	# Allow binding to non local addresses. Default no.
	#ip-transparent: yes

	# Allow binding to addresses that are down.  Default no.
	#ip-freebind: no

	# use the reuseport socket option for performance. Default no.
	#reuseport: yes

	# enable debug mode, does not fork daemon process into the background.
	debug-mode: no

	# listen on IPv4 connections
	do-ip4: yes

	# listen on IPv6 connections
	do-ip6: yes

	# port to answer queries on. default is 53.
	port: 53

jsierles · February 15, 2022, 8:51pm

Can you try with fly-global-services in the ip-address? If it doesn’t like hostnames, try using the equivalent value from /etc/hosts.

user115 · February 15, 2022, 9:08pm

Aha, I didn’t think to check /etc/hosts… I tried telling it to bind to fly-global-services and it said it was an unknown address, I’ll see what I can find in /etc/hosts.

Thanks so much for your help!

user115 · February 15, 2022, 9:20pm

This is what I see checking the container directly:

Connecting to top1.nearest.of.k-flydns.internal... complete
/ # cat /etc/hosts
127.0.0.1	localhost localhost.localdomain
::1		localhost localhost.localdomain

127.0.0.1	localhost localhost4

::1	localhost localhost6

# Address used for global traffic routing
172.19.1.139	fly-global-services

# Address in the 6PN private network for this app
fdaa:0:4d3d:a7b:87:f31:bf49:2	0f31bf49.vm.k-flydns.internal 0f31bf49 fly-local-6pn
/ # ifconfig -a
dummy0    Link encap:Ethernet  HWaddr 72:2D:A6:B0:AA:B3
          BROADCAST NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth0      Link encap:Ethernet  HWaddr DE:AD:35:F2:D3:72
          inet addr:172.19.1.138  Bcast:172.19.1.143  Mask:255.255.255.248
          inet6 addr: fe80::dcad:35ff:fef2:d372/64 Scope:Link
          inet6 addr: fdaa:0:4d3d:a7b:87:f31:bf49:2/112 Scope:Global
          inet6 addr: 2604:1380:4510:401:0:f31:bf49:1/127 Scope:Global
          UP BROADCAST RUNNING MULTICAST  MTU:1420  Metric:1
          RX packets:823 errors:0 dropped:0 overruns:0 frame:0
          TX packets:835 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:81126 (79.2 KiB)  TX bytes:81129 (79.2 KiB)

Looks like I’ll bind an address that isn’t assigned, so I’ll need to enable ip-transparent

I’ll give it a whirl and see if it works, thanks again.

user115 · February 15, 2022, 10:34pm

Hm, still no luck talking to it over UDP through the proxy. If I connect to the console I can run dig and things work.

It’d be neat if this worked, would be a simple path to BIND zonefile-compatible anycast DNS. Thanks for any advice.

my fly.toml:


kill_signal = "SIGINT"
kill_timeout = 5

[[services]]
  internal_port = 53
  protocol = "udp"

  [[services.ports]]
    port = 53

[[services]]
  internal_port = 53
  protocol = "tcp"

  [[services.ports]]
    port = 53

my (super ugly, ewwww) Dockerfile:


RUN apk --update --no-cache add nsd

LABEL maintainer="jw"

COPY /zones /etc/nsd/zones/
COPY /nsd.conf /etc/nsd/
COPY /includes /etc/nsd/includes/

RUN mkdir -p /var/lib/nsd
RUN chown nsd:nsd /var/lib/nsd
RUN chmod 0755 /var/lib/nsd

EXPOSE 53/tcp
EXPOSE 53/udp

#ENTRYPOINT [ "/usr/sbin/nsd","-d" ]

CMD [ "/usr/sbin/nsd", "-d" ]

my nsd.conf (first part):

	# Number of NSD servers to fork.  Put the number of CPUs to use here.
	#server-count: 2

	# uncomment to specify specific interfaces to bind (default are the
	# wildcard interfaces 0.0.0.0 and ::0).
	# For servers with multiple IP addresses, list them one by one,
	# or the source address of replies could be wrong.
	# Use ip-transparent to be able to list addresses that turn on later.
	# ip-address: 1.2.3.4
	# ip-address: 1.2.3.4@5678
	# ip-address: 12fe::8ef0
	ip-address: 172.19.1.139
	ip-address: ::

	# Allow binding to non local addresses. Default no.
	ip-transparent: yes

	# Allow binding to addresses that are down.  Default no.
	#ip-freebind: no

	# use the reuseport socket option for performance. Default no.
	reuseport: yes

	# enable debug mode, does not fork daemon process into the background.
	debug-mode: no

	# listen on IPv4 connections
	do-ip4: yes

	# listen on IPv6 connections
	do-ip6: yes

	# port to answer queries on. default is 53.
	port: 53

ifconfig on the container looks like:

/ # ifconfig -a
dummy0    Link encap:Ethernet  HWaddr C2:6C:93:05:68:78
          BROADCAST NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth0      Link encap:Ethernet  HWaddr DE:AD:E0:1F:AB:10
          inet addr:172.19.9.186  Bcast:172.19.9.191  Mask:255.255.255.248
          inet6 addr: 2604:1380:4510:500:0:6fd1:1352:1/127 Scope:Global
          inet6 addr: fe80::dcad:e0ff:fe1f:ab10/64 Scope:Link
          inet6 addr: fdaa:0:4d3d:a7b:85:6fd1:1352:2/112 Scope:Global
          UP BROADCAST RUNNING MULTICAST  MTU:1420  Metric:1
          RX packets:75 errors:0 dropped:0 overruns:0 frame:0
          TX packets:77 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:10088 (9.8 KiB)  TX bytes:9037 (8.8 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

/etc/hosts:

127.0.0.1	localhost localhost.localdomain
::1		localhost localhost.localdomain

127.0.0.1	localhost localhost4

::1	localhost localhost6

# Address used for global traffic routing
172.19.9.187	fly-global-services

# Address in the 6PN private network for this app
fdaa:0:4d3d:a7b:85:6fd1:1352:2	6fd11352.vm.k-flydns.internal 6fd11352 fly-local-6pn

kurt · February 15, 2022, 11:17pm

That internal IP changes each time your VM runs. You’ll need to generate your nsd.conf at boot time to get that IP in there.

Topic		Replies	Views
Inbound UDP not arriving Questions / Help	4	222	March 6, 2024
UDP isn't working	9	433	December 25, 2023
UDP port 53 stopped working(?) Questions / Help	5	382	June 10, 2022
Fly's sample echo app not working with UDP Questions / Help proxy	7	279	April 4, 2024
Issue with UDP connection (simple test with netcat) Questions / Help proxy	7	82	October 24, 2024

Inbound UDP not working

Related topics