Private networks: can't access .internal addresses | connection times out

1

I’m trying to access the private networking via a bash script based on the description over here: https://fly.io/docs/reference/services/#private-network-services.

My fly.toml includes:

[experimental]
  private_network = "true"

My Dockerfile’s entrypoint runs a shell script that prints the private networking info:

echo "get host configs: "$(cat /etc/resolv.conf)
# this prints - "get host configs:  nameserver fdaa::3" as expected.

echo "Names of 6PN private networking apps: "$(dig -t txt _apps.internal +short)
# this prints - "Names of 6PN private networking apps:  ; <<>> DiG 9.11.20-RedHat-9.11.20-5.el8 <<>> -t txt _apps.internal +short ;; global options: +cmd ;; connection timed out; no servers could be reached"

echo "Deployed regions: "$(dig -t txt ${appname}.internal +short)
# this prints - "Deployed regions:  ; <<>> DiG 9.11.20-RedHat-9.11.20-5.el8 <<>> -t txt ${appname}.internal +short ;; global options: +cmd ;; connection timed out; no servers could be reached"

echo "IPv6 address of app instances in any region: "$(getent hosts global.${appname}.internal | awk '{ print $1 }')
# this prints - "IPv6 address of app instances in any region:"

It seems there’s a connection time out and this shouldn’t be the expected behavior. I also read this response from @thomas explaining how private addresses are injected prior to the boot sequence: Incoming! Private Networks and WireGuard VPNs.

Also wondering, is there a way to troubleshoot by exec’ing into the app? I don’t any obvious mention of this in the docs.

I’ve also tried out the privatenet example repo - fly-examples/privatenet. And while it deploys successfully, the endpoint / times out. I’m not sure if the two are related.

2

Exec isn’t publicly available yet, but if you shared an instance ID we can go poke around for you!

This does sound like you’ve hit a bug, but you might also try passing dig the @fdaa::3 option to dig and see if you get any different result.

3

Also, what region was this running in?

4

Thanks for the prompt response. This was deployed in sin region. I’ve destroyed the app, will re-create again and let you know.

5

Update here: When I deploy in iad region, I can see the 6PN address of the app instance everytime I tried. Though here, $(dig -t txt _apps.internal +short) doesn’t reliably get the app name. I have seen get it sometimes and miss it sometimes :thinking:

I had earlier tried sin and hkg regions earlier, where in both cases, I couldn’t get the 6PN addresses to resolve (the original issue that I posted).

6

Hey there! You were definitely right about the sin workers; the host you landed on was new and hit a corner case in the DNS installer. Thanks for spotting that.

I’m not seeing anything janky on hkg right now.

7

dig -t txt _apps.internal +short can take a few seconds to show up since it has to propagate through our system.