WireGuard tunnel: can't resolve <app>.internal etc.

I’m trying to create a WireGuard tunnel to the 6PN private network, but DNS only works partially.

Also, fly wireguard status outputs “Error: You hit a Fly API error with request ID: 01H1Y3BVHZF39MJ7V3VABEG4MB-ord” the first time, and later “Error: upstream service is unavailable”.

The following resolved: <app>.fly.dev, _apps.internal
The following didn’t produce any results: <app>.internal, _instances.internal, _peer.internal

I tried the same steps via https://fly.io/terminal and the results were a little different: <app>.internal and _instances.internal resolved (only after I set up WireGuard), and _peer.internal didn’t produce any results.

Finally, when running the same DNS queries from the demo app’s VM, everything resolved except _peer.internal.

(Look for the lines marked “PROBLEM” below.)

# OS and flyctl versions
$ lsb_release -d
Description:    Debian GNU/Linux 11 (bullseye)
$ fly version
flyctl v0.1.24 linux/amd64 Commit: 86409d36fda482b419410a4af3113d12537a06d7 BuildDate: 2023-06-01T11:52:45Z

# install wireguard
$ sudo apt install wireguard openresolv
...
Unpacking openresolv (3.12.0-1) ...
Unpacking wireguard-tools (1.0.20210223-1) ...
Unpacking wireguard (1.0.20210223-1) ...
...

# demo app
$ mkdir app && cd app
$ cat > Dockerfile <<"EOF"
FROM debian:11
CMD sleep infinity
EOF
$ fly launch
...
? Choose a region for deployment: Chicago, Illinois (US) (ord)
...
Platform: machines
...
# disable auto-stop
$ sed -E -i '/^  auto_stop_machines = true$/ s/true/false/' fly.toml
# get the app name
$ app="$(sed -E -n 's/^app = "([^"]*)"$/\1/p' fly.toml)"
$ fly deploy --ha=false
...
$ fly ips list
VERSION IP                      TYPE            REGION  CREATED AT
v6      2a09:...                public          global  ...s ago
v4      66.2...                 public (shared)

# try some DNS queries without wireguard
$ dig +short aaaa $app.fly.dev
2a09:...
$ dig +noall +answer _apps.internal txt # https://fly.io/docs/reference/private-networking/#testing-the-tunnel
# no output (as expected)
$ dig +short aaaa $app.internal
# no output (as expected)

# start wireguard
$ fly wireguard create
...
? Filename to store WireGuard configuration in, or 'stdout':  basic.conf
$ sudo cp basic.conf /etc/wireguard/basic.conf
$ sudo wg-quick up basic
[#] ip link add basic type wireguard
[#] wg setconf basic /dev/fd/63
[#] ip -6 address add fdaa:...:202/120 dev basic
[#] ip link set mtu 1420 up dev basic
[#] resolvconf -a basic -m 0 -x
[#] ip -6 route add fdaa:...::/48 dev basic

# check the status of wireguard
$ sudo wg
interface: basic
  public key: ...
  private key: (hidden)
  listening port: 58413

peer: ...
  endpoint: 43.2...:51820
  allowed ips: fdaa:...::/48
  latest handshake: 2 seconds ago
  transfer: 92 B received, 180 B sent
  persistent keepalive: every 15 seconds
$ sudo systemctl status wg-quick@basic
● wg-quick@basic.service - WireGuard via wg-quick(8) for basic
     Loaded: loaded (/lib/systemd/system/wg-quick@.service; disabled; vendor preset: enabled)
     Active: inactive (dead)
       Docs: ...
# not sure why it shows as "inactive"
$ sudo journalctl
...
... systemd-udevd[849]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
... kernel: wireguard: WireGuard 1.0.0 loaded. See www.wireguard.com for information.
... kernel: wireguard: Copyright (C) 2015-2019 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
... systemd-udevd[849]: Using default interface naming scheme 'v247'.
...
$ cat /etc/resolv.conf
# Generated by resolvconf
nameserver fdaa:...::3

$ fly wireguard status
? Select peer: <select interactively>
Error: upstream service is unavailable
# PROBLEM (also get this for peer connections generated automatically by flyctl)
#
# Output with FLY_LOG=debug:
#
# DEBUG --> POST https://api.fly.io/graphql
#
# {
#   "query": "query($slug: String!, $name: String!) { organization(slug: $slug) { wireGuardPeer(name: $name) { gatewayStatus } } }",
#   "variables": {
#     "name": "interactive-...",
#     "slug": "personal"
#   }
# }
#
# DEBUG {}
# DEBUG <-- 200 https://api.fly.io/graphql (311.56ms)
#
# {
#   "data": {
#     "organization": null
#   },
#   "errors": [
#     {
#       "message": "upstream service is unavailable",
#       "locations": [
#         {
#           "line": 1,
#           "column": 98
#         }
#       ],
#       "path": [
#         "organization",
#         "wireGuardPeer",
#         "gatewayStatus"
#       ],
#       "extensions": {
#         "code": "SERVICE_UNAVAILABLE"
#       }
#     }
#   ]
# }

# try some DNS queries with wireguard
$ dig +short aaaa $app.fly.dev
2a09:...
$ dig +noall +answer _apps.internal txt
_apps.internal.         5       IN      TXT     "fly-builder-...,<app>"
$ dig +noall +answer _instances.internal txt
# no output (PROBLEM)
$ dig +short txt _peer.internal
# no output (PROBLEM)
$ dig +short aaaa $app.internal
# no output (PROBLEM)
$ ping -4 $app.internal
ping: <app>.internal: No address associated with hostname
$ ping -6 $app.internal
ping: <app>.internal: No address associated with hostname

# try some DNS queries from the VM
$ fly ssh console
# apt update && apt install dnsutils
# dig +short aaaa $FLY_APP_NAME.fly.dev
2a09:...
# dig +noall +answer _apps.internal txt
_apps.internal.         5       IN      TXT     "fly-builder-...,<app>"
# dig +noall +answer _instances.internal txt
_instances.internal.    5       IN      TXT     "instance=...,app=<app>,ip=fdaa:...:2,region=ord"
# dig +short txt _peer.internal
# no output (PROBLEM?)
# dig +short aaaa $FLY_APP_NAME.internal
fdaa:...:2

Having similar issue. And this thread is probably related - Postgres connection via Wireguard + TablePlus error: could not translate host name

The error I’m getting:

Error You hit a Fly API error with request ID: 01H1YK1V14HP9V95SG99CP86SH-nrt
1 Like

This works for me now, so I’m closing the thread.

With _peer.internal, the reason it didn’t work is that the default generated names (interactive-*) are filtered out. Using a custom name makes _peer.internal work.

Regarding fly wireguard status, it was recently removed.

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.