Unable to connect to Postgres via proxy / no response from wireguard gateway

Hey there, I tried to establish a tunnel to my postgres db this morning with the command that worked so far:

fly proxy 5433 -a my-database-name

I will always receive this error:

Error: tunnel unavailable for organization ORGNAME: Error contacting Fly.io API when probing "ORGNAME": timed out (context deadline exceeded)

I then found this question in the forum and tried to reset my wireguard connection with fly wg reset, but it didn’t help.

I also noticed that fly wg list produces a huge list with 74 entries to regions all over the place. That does not seem normal? We are a small org with two devs.

This is the result of fly doctor. Same for my colleague, so I doubt it is related to my machine.

🤷 ❯ fly doctor
Testing authentication token... PASSED
Testing flyctl agent... PASSED
Testing local Docker instance... Nope
Pinging WireGuard gateway (give us a sec)... FAILED
(Error: ping gateway: no response from gateway received)

We can't establish connectivity with WireGuard for your personal organization.

WireGuard runs on 51820/udp, which your local network may block.

If this is the first time you've ever used 'flyctl' on this machine, you
can try running 'flyctl doctor' again.

If this was working before, you can ask 'flyctl' to create a new peer for
you by running 'flyctl wireguard reset'.

If your network might be blocking UDP, you can run 'flyctl wireguard websockets enable',
followed by 'flyctl agent restart', and we'll run WireGuard over HTTPS.

Any idea what is going on?

we have the same issue. yesterday everything was working, not anymore (and yes, I have tried the same steps, nothing was helping)

I think it is not postgres related, because all tunnels are not working (like ssh to app)

Thank you for sharing. Yes, I also think it is wireguard related since the ping to the Wireguard gateway also fails for fly doctor.

I also noticed two other things:

  • flyctl was upgraded to 0.3.129. Maybe that introduced the issue?
  • There are a ton of “Flynthetics” access tokens in the dashboard that I did not create. They all expire in an hour.

Would be great if someone from the Fly Team could look into this. We are about to deploy a large release and I will have to postpone it until we can ssh into our machines again.

Same issue here. Maybe it’s unrelated, but I’ve updated to Mac OS 15.5 yesterday, since this morning I receive this error:

Error: tunnel unavailable for organization personal: Error contacting Fly.io API when probing “personal”: timed out (context deadline exceeded)

Interesting. Same here. I did the update to Mac OS 15.5 this morning. When I tried to connect to our Postgres after I started seeing this issue.

I can take a look at this, what region is the wireguard peer created in? (you can check in ~/.fly/config.yml)

Thanks. So I have fra in config.yml, but my machines run in ams. I’m not sure why this is the case. When I set everything up I think Frankfurt wasn’t available. But everything worked until yesterday.

thanks, I can reproduce in fra region

(the gateway gets configured for the region closest to you, rather than the region in which your machines run. that’s normal)

Same here! Also region fra.

Similar issue here, region is syd

Disabeling websockets removes the error from fly doctor but connection still doesn’t work

🤷 ❯ fly wg websockets disable && fly agent restart
🤷 ❯ fly doctor
Testing authentication token... PASSED
Testing flyctl agent... PASSED
Testing local Docker instance... Nope
Pinging WireGuard gateway (give us a sec)... PASSED
Testing WireGuard DNS... PASSED
Testing WireGuard Flaps... PASSED
No app provided; skipping app specific checks
🤷 ❯ fly proxy 5433 -a MYAPP --debug --verbose
Error: tunnel unavailable for organization layers: Error contacting Fly.io API when probing "layers": timed out (context deadline exceeded)
Stacktrace:
goroutine 1 [running]:
runtime/debug.Stack()
	runtime/debug/stack.go:26 +0x64
github.com/superfly/flyctl/internal/cli.printError(0x14000499d60, 0x140007f5bfd, 0x140005e6608, {0x1069cc660, 0x1400043e280})
	github.com/superfly/flyctl/internal/cli/cli.go:185 +0x44c
github.com/superfly/flyctl/internal/cli.Run({0x1069f2fb0?, 0x1400019f000?}, 0x14000499d60, {0x1400004c240, 0x6, 0x6})
	github.com/superfly/flyctl/internal/cli/cli.go:118 +0x9c4
main.run()
	github.com/superfly/flyctl/main.go:47 +0x170
main.main()
	github.com/superfly/flyctl/main.go:26 +0x20

hey, it should be fixed now, could you try again (with websockets enabled is ok)?

hey, it should be fixed now, could you try again (with websockets enabled is ok)?

It works now again on our side (AMS), thank you.

Hey, works again for FRA with and without websockets. Thanks for taking care of this!