Wireguard tunnel only works for like 20 minutes. Then it stops

[armando]

I´ve been using wireguard tunnels for a while. But for some reason a week ago I realized that my tunnels were not working.

What I do:

fly wireguard create

Choose my organization, then, type a name to create a file.
Import the tunnel and activate it.
It works

After like 20 minutes, it stops working and can’t connect again. I checked the logs and it times out waiting for handshake response

Does anyone has any idea what could it be? Then, if I go and create a new tunnel it works

Thanks in advance.

PS: I am using Windows 11

Logs:

2024-06-17 18:17:13.203823: [TUN] [flyio-wireguard] Starting WireGuard/0.5.3 (Windows 10.0.22631; amd64)
2024-06-17 18:17:13.203823: [TUN] [flyio-wireguard] Watching network interfaces
2024-06-17 18:17:13.204853: [TUN] [flyio-wireguard] Resolving DNS names
2024-06-17 18:17:13.452304: [TUN] [flyio-wireguard] Creating network adapter
2024-06-17 18:17:13.490587: [TUN] [flyio-wireguard] Using existing driver 0.10
2024-06-17 18:17:13.494120: [TUN] [flyio-wireguard] Creating adapter
2024-06-17 18:17:13.559903: [TUN] [flyio-wireguard] Using WireGuardNT/0.10
2024-06-17 18:17:13.559903: [TUN] [flyio-wireguard] Enabling firewall rules
2024-06-17 18:17:13.547808: [TUN] [flyio-wireguard] Interface created
2024-06-17 18:17:13.561462: [TUN] [flyio-wireguard] Dropping privileges
2024-06-17 18:17:13.561462: [TUN] [flyio-wireguard] Setting interface configuration
2024-06-17 18:17:13.561976: [TUN] [flyio-wireguard] Peer 1 created
2024-06-17 18:17:13.564041: [TUN] [flyio-wireguard] Sending keepalive packet to peer 1 (redacted-ip-1:51820)
2024-06-17 18:17:13.564041: [TUN] [flyio-wireguard] Sending handshake initiation to peer 1 (redacted-ip-1:51820)
2024-06-17 18:17:13.564041: [TUN] [flyio-wireguard] Interface up
2024-06-17 18:17:13.564041: [TUN] [flyio-wireguard] Monitoring MTU of default v4 routes
2024-06-17 18:17:13.565330: [TUN] [flyio-wireguard] Setting device v4 addresses
2024-06-17 18:17:13.566743: [TUN] [flyio-wireguard] Monitoring MTU of default v6 routes
2024-06-17 18:17:13.567372: [TUN] [flyio-wireguard] Setting device v6 addresses
2024-06-17 18:17:13.568786: [TUN] [flyio-wireguard] Startup complete
2024-06-17 18:17:18.599035: [TUN] [flyio-wireguard] Handshake for peer 1 (redacted-ip-1:51820) did not complete after 5 seconds, retrying (try 2)
2024-06-17 18:17:18.599035: [TUN] [flyio-wireguard] Sending handshake initiation to peer 1 (redacted-ip-1:51820)
2024-06-17 18:17:23.667966: [TUN] [flyio-wireguard] Sending handshake initiation to peer 1 (redacted-ip-1:51820)
2024-06-17 18:17:28.739699: [TUN] [flyio-wireguard] Handshake for peer 1 (redacted-ip-1:51820) did not complete after 5 seconds, retrying (try 2)
2024-06-17 18:17:28.739699: [TUN] [flyio-wireguard] Sending handshake initiation to peer 1 (redacted-ip-1:51820)
2024-06-17 18:17:33.857072: [TUN] [flyio-wireguard] Handshake for peer 1 (redacted-ip-1:51820) did not complete after 5 seconds, retrying (try 2)
2024-06-17 18:17:33.857072: [TUN] [flyio-wireguard] Sending handshake initiation to peer 1 (redacted-ip-1:51820)
2024-06-17 18:17:38.862401: [TUN] [flyio-wireguard] Sending handshake initiation to peer 1 (redacted-ip-1:51820)
2024-06-17 18:17:43.886618: [TUN] [flyio-wireguard] Handshake for peer 1 (redacted-ip-1:51820) did not complete after 5 seconds, retrying (try 2)
2024-06-17 18:17:43.886618: [TUN] [flyio-wireguard] Sending handshake initiation to peer 1 (redacted-ip-1:51820)
2024-06-17 18:17:49.002654: [TUN] [flyio-wireguard] Handshake for peer 1 (redacted-ip-1:51820) did not complete after 5 seconds, retrying (try 2)
2024-06-17 18:17:49.002654: [TUN] [flyio-wireguard] Sending handshake initiation to peer 1 (redacted-ip-1:51820)
2024-06-17 18:17:54.169796: [TUN] [flyio-wireguard] Handshake for peer 1 (104.225.8.204:51820) did not complete after 5 seconds, retrying (try 2)

.....

[armando]

Bumping up this thread…

Does anyone have any insights?

[lubien]

Hey there. Thanks for bumping this. Since I don’t have much context on wg internals I’ve raised this to our Product Security team.

[armando]

@lubien thanks for the response. Hoping to hear back soon!

[thomas]

Armando: you have nerd-sniped us. The WireGuard nerds here are all looking at this.

Sorry if I missed this, but can you tell us the name of the WireGuard peer that you loaded that died in 20 minutes?

Did you create it within the last week, or is this one you created a long time ago?

Do you know if you’re using current flyctl?

We did some minor surgery on how we track peers about a month and a half ago and we want to rule some things out.

If you’re seeing this, other people are seeing it too, so thank you for calling this out.

[armando]

Well …

Now that you mentioned it, I checked fly version and it was in fact outdated. Weird, cause I had it set up for auto-update. So I updated to the latest version and I am more than 30 minutes using the tunnel and hasn’t dropped the connection.

I notice that new created peers starts with static-.... the failing ones started with interactive-...

I am going to use it a little further and provide an update later today. But I am fairly hopeful that updating the CLI solved it.

Answering your other question: The older tunnels stopped working as well.

Thanks

[thomas]

We’ve reproduced this problem. Updating flyctl should have fixed it for you (newer flyctl generates peers from flyctl wireguard create with slightly different metadata that doesn’t trigger garbage collection). If you’re stuck with older flyctl somewhere, we can give you an immediate workaround; in the meantime, we’re looking into a global fix on our side.

Thanks for calling this out!

[armando]

Well, thanks for the reply.

After a few hours working fine and not closing. I can say, on my case, updating to the newer version of flyctl it created the right peers with the right configuration. Marking this as the solution.

Thanks for the help

[scriptjs]

Thanks @armando. Same issue with MacOS. All my old tunnels stopped working and prevented dns on my local system when activated. They worked before I left work for a few weeks ago. Last upgrade on flyctl was 0.2.42 that was released April 24. Mine also were named with “interactive-" while new begin with "static-”