Unable to deploy new version - WebSocket dial: expected handshake response status code 101 but got 403

Questions / Help Build debugging
1

Since this morning I am running into an issue which seemingly has to do with Wireguard.

Deploying returns:

➜ flyctl deploy
==> Verifying app config
Validating /Users/<my_name>/Projects/throwdown/fly.toml
Platform: machines
✓ Configuration is valid
--> Verified app config
==> Building image
WARN Failed to start remote builder heartbeat: failed building options: websocket: failed to WebSocket dial: expected handshake response status code 101 but got 403

Error: failed to fetch an image or build from source: error connecting to docker: failed building options: websocket: failed to WebSocket dial: expected handshake response status code 101 but got 403

Doctor returns:

➜ flyctl doctor
Testing authentication token... PASSED
Testing flyctl agent... PASSED
Testing local Docker instance... Nope
Pinging WireGuard gateway (give us a sec)... FAILED
(Error: ping gateway: pinger: websocket: failed to WebSocket dial: expected handshake response status code 101 but got 403)

We can't establish connectivity with WireGuard for your personal organization.

WireGuard runs on 51820/udp, which your local network may block.

If this is the first time you've ever used 'flyctl' on this machine, you
can try running 'flyctl doctor' again.

If this was working before, you can ask 'flyctl' to create a new peer for
you by running 'flyctl wireguard reset'.

If your network might be blocking UDP, you can run 'flyctl wireguard websockets enable',
followed by 'flyctl agent restart', and we'll run WireGuard over HTTPS.

I have tried the steps mentioned above but it fails at the reset:

➜ flyctl wireguard reset
automatically selected personal organization: <my_name>
Error: websocket: failed to WebSocket dial: expected handshake response status code 101 but got 403

fly status also fails:

➜ fly status
Error: failed to list active VMs: Get "https://api.machines.dev/v1/apps/<app_name>/machines": tls: failed to verify certificate: x509: “api.machines.dev” certificate is not trusted

Other steps I took without success:

  • Removing the builder
  • Removing all Wireguard connections
  • Restarting the agent (fly agent restart)
  • Deleting and reinstalling the flyctl cli
  • Check any running (stuck) fly cli processes
  • Connect to another network
  • Restart Macbook

The issue pages (global and personal) don’t show anything as well.

Based on what I have read on other topics it might have to do with this being the second month on the Hobby Plan but that’s just a hunch.

Thanks in advance for the help!

2

Found the issue, it was the firewall of the network blocking the traffic to fly APIs.
I initially thought that was not the issue because I also tried a hotspot on my phone but my phone was connected to the same network.

1 Like
3

@kay How did you even figure that out! Thanks anyways you saved me so much time and hair pulling

4

I tried visiting one of the APIs via the webbrowser and got an alert from the firewall on the network