I’ve been trying to connect to a PostgreSQL cluster deployed on Fly, but got this error:
$ flyctl proxy 15432:5432 -a xxx
Opening a wireguard tunnel to personal 🌎Error: tunnel unavailable for organization personal: failed probing "personal": context deadline exceeded
Looking around I found that there’s an issue with my agent being unable to connect to your servers. Even enabling websockets via flyctl wg websockets enable (and restarting the agent) didn’t change it.
Doctor gives me:
$ flyctl doctor 24-Oct-2023 15:30:42
Testing authentication token... PASSED
Testing flyctl agent... PASSED
Testing local Docker instance... PASSED
Pinging WireGuard gateway (give us a sec)... FAILED
(Error: ping gateway: no response from gateway received)
We can't establish connectivity with WireGuard for your personal organization.
WireGuard runs on 51820/udp, which your local network may block.
If this is the first time you've ever used 'flyctl' on this machine, you
can try running 'flyctl doctor' again.
If this was working before, you can ask 'flyctl' to create a new peer for
you by running 'flyctl wireguard reset'.
If your network might be blocking UDP, you can run 'flyctl wireguard websockets enable',
followed by 'flyctl agent restart', and we'll run WireGuard over HTTPS.
Tried that, but didn’t change anything.
So I started the agent with LOG_LEVEL=debug fly agent run and got this:
2023/10/24 15:30:37.446102 #19 connected ...
2023/10/24 15:30:37.446199 #19 <- ( 14) "probe personal"
2023/10/24 15:30:37.446213 srv probing "personal" ...
DEBUG: (fly-ssh) 2023/10/24 15:30:37 peer(KC8f…tIDs) - Sending handshake initiation
2023/10/24 15:30:42.447253 #19 -> ( 58) "8\x00err failed probing \"personal\": context deadline exceeded"
2023/10/24 15:30:42.447458 #19 dropped.
DEBUG: (fly-ssh) 2023/10/24 15:30:42 peer(KC8f…tIDs) - Handshake did not complete after 5 seconds, retrying (try 2)
DEBUG: (fly-ssh) 2023/10/24 15:30:42 peer(KC8f…tIDs) - Sending handshake initiation
DEBUG: (fly-ssh) 2023/10/24 15:30:47 peer(KC8f…tIDs) - Handshake did not complete after 5 seconds, retrying (try 3)
DEBUG: (fly-ssh) 2023/10/24 15:30:47 peer(KC8f…tIDs) - Sending handshake initiation
DEBUG: (fly-ssh) 2023/10/24 15:30:52 peer(KC8f…tIDs) - Handshake did not complete after 5 seconds, retrying (try 4)
DEBUG: (fly-ssh) 2023/10/24 15:30:52 peer(KC8f…tIDs) - Sending handshake initiation
DEBUG: (fly-ssh) 2023/10/24 15:30:58 peer(KC8f…tIDs) - Handshake did not complete after 5 seconds, retrying (try 5)
DEBUG: (fly-ssh) 2023/10/24 15:30:58 peer(KC8f…tIDs) - Sending handshake initiation
DEBUG: (fly-ssh) 2023/10/24 15:31:03 peer(KC8f…tIDs) - Handshake did not complete after 5 seconds, retrying (try 6)
DEBUG: (fly-ssh) 2023/10/24 15:31:03 peer(KC8f…tIDs) - Sending handshake initiation
...
DEBUG: (fly-ssh) 2023/10/24 15:31:39 peer(KC8f…tIDs) - Sending handshake initiation
2023/10/24 15:31:40.618032 srv (re-)connecting to wss://fra2.gateway.6pn.dev:443/
2023/10/24 15:31:41.511975 srv resetting connection due to error: failed to write msg: WebSocket closed: received close frame: status = StatusNormalClosure and reason = ""
DEBUG: (fly-ssh) 2023/10/24 15:31:44 peer(KC8f…tIDs) - Handshake did not complete after 5 seconds, retrying (try 14)
DEBUG: (fly-ssh) 2023/10/24 15:31:44 peer(KC8f…tIDs) - Sending handshake initiation
2023/10/24 15:31:46.511813 srv resetting connection due to error: failed to write msg: WebSocket closed: received close frame: status = StatusNormalClosure and reason = ""
DEBUG: (fly-ssh) 2023/10/24 15:31:49 peer(KC8f…tIDs) - Handshake did not complete after 5 seconds, retrying (try 15)
DEBUG: (fly-ssh) 2023/10/24 15:31:49 peer(KC8f…tIDs) - Sending handshake initiation
2023/10/24 15:31:51.512002 srv resetting connection due to error: EOF
DEBUG: (fly-ssh) 2023/10/24 15:31:54 peer(KC8f…tIDs) - Handshake did not complete after 5 seconds, retrying (try 16)
DEBUG: (fly-ssh) 2023/10/24 15:31:54 peer(KC8f…tIDs) - Sending handshake initiation
DEBUG: (fly-ssh) 2023/10/24 15:32:00 peer(KC8f…tIDs) - Handshake did not complete after 5 seconds, retrying (try 17)
DEBUG: (fly-ssh) 2023/10/24 15:32:00 peer(KC8f…tIDs) - Sending handshake initiation
DEBUG: (fly-ssh) 2023/10/24 15:32:05 peer(KC8f…tIDs) - Handshake did not complete after 5 seconds, retrying (try 18)
DEBUG: (fly-ssh) 2023/10/24 15:32:05 peer(KC8f…tIDs) - Sending handshake initiation
DEBUG: (fly-ssh) 2023/10/24 15:32:10 peer(KC8f…tIDs) - Handshake did not complete after 5 seconds, retrying (try 19)
DEBUG: (fly-ssh) 2023/10/24 15:32:10 peer(KC8f…tIDs) - Sending handshake initiation
DEBUG: (fly-ssh) 2023/10/24 15:32:15 peer(KC8f…tIDs) - Handshake did not complete after 5 seconds, retrying (try 20)
DEBUG: (fly-ssh) 2023/10/24 15:32:15 peer(KC8f…tIDs) - Sending handshake initiation
DEBUG: (fly-ssh) 2023/10/24 15:32:20 peer(KC8f…tIDs) - Handshake did not complete after 20 attempts, giving up
LOG2023/10/24 15:32:52.511703 srv resetting connection due to error: EOF
2023/10/24 15:32:52.511987 srv resetting connection due to error: EOF
2023/10/24 15:32:57.512822 srv resetting connection due to error: EOF
2023/10/24 15:33:00.618848 srv (re-)connecting to wss://fra2.gateway.6pn.dev:443/
2023/10/24 15:33:02.512867 srv resetting connection due to error: failed to write msg: WebSocket closed: received close frame: status = StatusNormalClosure and reason = ""
I suppose something might be wrong network-wise on my side, but I haven’t figured out what it could be. Trying on a different computer got me the same results though.
Any ideas?
Thanks in advance,
Oliver