Wireguard private networking with distinct networks

I have a scenario where I’d like to use fly to set up cloud instances that my customers can access via Wireguard private networks. Each user will need their own network, as my customers need to be isolated from one another.

From conversations with @kurt, it sounds like this functionality is almost available, but not 100% implemented. What would be a timeline for having this support available?

The plumbing is all here for this! I think it’s a quick thing to expose through GraphQL. We should be able to get it in early next week.

3 Likes

Hi @kurt, just wanted to check if this still on your radar.

It is! The GraphQL mutation for creating a wireguard peer takes a network parameter now. If you create an app with a network name, you can then create a peer in that same network.

This isn’t in flyctl yet but it’s probably not so hard to add!

1 Like

Awesome! Availability in the graphql API is good enough for me.

Good deal. It’s pretty simple to generate the key pair you need for a peer,flyctl does it in Go.

Just to clarify, the following works?

  1. Create an app, specify network to be anything
  2. Create a WireGuard peer, set network to be the same as the app?

This isolates the WireGuard peer to only apps that share the network? Is network a globally-unique value or an organizationally-unique value?

Is there a way to connect an app to more than one network? For example, I want to isolate client connections to their own instances running in Fly that I manage and I also want to be able to connect via WireGuard without network isolation.

If I omit the network on the WireGuard peer does that provide global access to all apps, even if the app specifies a network? What about Postgres? Does that mean that if I have Postgres and a network specified on the app, the app cannot communicate with an untagged Postgres HA in Fly (without network)?

I ask because I want to establish a WireGuard connection between the managed app instances and agents that communicate with the cloud instance. Right now, the agent checks in with a server, but I’d prefer to go the other way when possible to avoid polling or streaming commands. My current implementation uses user-land WireGuard from Go and establishes a peer for each app instance in the cluster for a client, and each agent as a peer.

This should work, but if WireGuard peering is already available as part of Fly I won’t need to build both ends. I don’t want to abuse the Fly implementation for fear of it breaking suddenly, but if this is an intended use case I’d love to use it.

Yep, your #1 and #2 work. network is unique per organization. If you omit network you get a default network that’s still isolated from the others (it’s almost the same as specifying network: <default>).

There’s not (yet) a built in way to put an app in more than one network. Your userland go option might be your best bet, but you can also just create a normal Wireguard interface in your VM as a “peer” to any network you’d need. It’s baked into the kernel. We shipped a little Wireguard token API to help with this kind of setup: WireGuard Token API

@kurt Just to clarify. My understanding is that apps can only talk to eachother through the internal network if they are in the same network, right? So I can isolate groups of apps using the same network name this way?

Thanks, yeah that’s what I was wondering. I use wgctrl now, which picks the right implementation to use based on the host. If that can properly determine wg in the kernel of Fly, that’s what it will use when running on Fly to avoid userland wg. If that’s not the case I’ll have to get creative with reworking it to, when deployed on Fly, use the Fly API.

I will stick to creating my own peers for now and can revisit if the ability to attach multiple networks becomes available. In a perfect world, this will all be managed automatically and the clients hosting will never know it happens. :slight_smile:

@Jens that’s exactly right!