Jack into your private network with WireGuard

Fresh Produce
1

I’m working on a series of Fly.io video tutorials so you can learn how to use all of the secret or indirect features on the platform. Today’s video is about accessing your org’s private network with WireGuard:

Jack into your private network with WireGuard

What do you think? Do you have any features you’d like to see covered? I’d love to hear your feedback so that you can learn all of the fun things that Fly.io has to offer.

For those of you that learn better from reading, I’ve copied my script here with some tweaks to make it flow better in written form.

Jack into your private network with WireGuard

Every Fly.io organization comes with a private network that lets all your apps connect to each other. This is super convenient when you need to have microservices call eachother’s endpoints or use Flycast to let your private apps turn off when you’re not using them. However, this isn’t just limited to your apps. You can jack into this network with WireGuard.

Today I’ll show you how to create a WireGuard peer to your private network and connect to it so that you can access it from anywhere.

Prerequisites

In order to get started, you need to have the following:

Steps

When you create a WireGuard peer, you need the following information:

  • The organization you want to create the peer in, such as your personal organization.
  • The fly.io region that’s closest to you.
  • The name of the peer, such as your computer’s hostname
  • A filename to save the configuration to

You can figure out your list of organizations with fly orgs list:

$ fly orgs list

You can figure out which region is nearest you with fly platform regions:

$ fly platform regions

With all this in mind, let’s assemble the command. Start with:

$ fly wireguard create

I want to create this in my personal organization, so I’ll enter in personal for the organization name.

$ fly wireguard create personal

I’m in Ottawa, so I’m using the Montreal region.

$ fly wireguard create personal yul

My computer’s hostname is Camellia, so I’ll use that as the peer name.

$ fly wireguard create personal yul camellia

Finally I want to save this as camellia.conf so that WireGuard can load it.

$ fly wireguard create personal yul camellia camellia.conf

Then I run the command and once it’s done I open up the WireGuard app.

Import the tunnel from the configuration file and then turn it on. macOS may prompt if you want the WireGuard app to manage VPN connections. If it does, hit accept, otherwise you won’t be able to get into your network.

To test it, ping _api.internal (NOTE: on macOS you need to run ping6 _api.internal because it’s an IPv6 address):

$ ping6 _api.internal -c4
PING6(56=40+8+8 bytes) fdaa:3:9018:a7b:9285:0:a:602 --> fdaa:3:9018::3
16 bytes from fdaa:3:9018::3, icmp_seq=0 hlim=64 time=9.741 ms
16 bytes from fdaa:3:9018::3, icmp_seq=1 hlim=64 time=49.103 ms
16 bytes from fdaa:3:9018::3, icmp_seq=2 hlim=64 time=97.667 ms
16 bytes from fdaa:3:9018::3, icmp_seq=3 hlim=64 time=14.726 ms

--- _api.internal ping6 statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 9.741/42.809/97.667/35.111 ms

To test this, I’ll fire up an instance of Ollama that I have on my private network.

export OLLAMA_HOSTNAME=http://xe-ollama.flycast
ollama run llama3 "Why is the sky blue? Explain in a single sentence."

And then the model will reply with something like this:

The sky appears blue because of a phenomenon called Rayleigh scattering, where shorter wavelengths of light (like blue and violet) are scattered more than longer wavelengths (like red and orange) by tiny molecules of gases like nitrogen and oxygen in the Earth’s atmosphere.

And there you go! Hope this helps give you ideas on how you can bend your network to your will. Have a good day all.

5 Likes
2

Is this appropriate for accessing a web server running internally (but not exposed to the internet) in my application?
I have followed these instructions but was unable to access anything at http://my-app.flycast:9000
my-app.flycast does not resolve on my machine after running the the above commands. Wire is setup and ping6 _api.internal -c4 works.
I can also ssh to the machine and check for port 9000 listening from a shell.

3

hey! Did you also try http://my-app.flycast - Flycast automatically resolves to the port your app is listening on :slight_smile:

4

About private network, I’m trying to connect a Meilisearch app to my webapp, all inside the same org. The Meilisearch is only for internal so using fly-local-6pn. It’s up an listening on 7700.
In the my webapp toml I’ve set [env] as MEILI_HOST = http://meili-app.internal:7700
Not connecting. What am I doing wrong?

5
  1. Use .flycast instead
  2. Double check you do indeed have a private ipv6 address
  3. Make sure your app binds to [::]
1 Like
6

I have changed the bind address to “[::1]:8090” but there is also a ipv4 service running on “0.0.0.0:8080” this service is exposed to the internet. Both are running in the same docker container.
Using http://myapp.flycast does not resolve. I assume this is because it points to service exposed on port 8080?

7

@highway900 whats the public port for your service? If the public port isn’t 80 then you need to add the port in your url, e.g. if the public port is 8080 then http://myapp.flycast:8080

8

So I have tried to access this service on http://my-app.flycast:8090 the service has a bind to [::1]:8090
If I run wireguard and ping6 _api.internal -c4 I get a ping back, but if I curl http://my-app.flycast I get curl: (6) Could not resolve host: my-app.flycast
I’m sure I am just doing something wrong or my use case is not possible (maybe?)

  • I have a single docker deployment with 2 services listening on 2 different ports
  • 1 service is public on 8080 and I can access this publicly via https://my-app.fly.dev
  • 1 service is NOT public and listens on port 8090
  • I need private/secure web access to the admin panel on 8090 (e.g. via the proxy, or wireguard etc)

Is this even possible?

9

Hi… It is, but you might not have a Flycast address assigned yet.

What does fly ips list show?

Edit: Flycast is not necessarily the right choice for a private admin panel, but the absence of such an address would explain the error message you’re seeing.

10

Thank @mayailurus I have 2 listed IPs a v4 and a v6, both are public.

I am very open to other solutions if using flycast is not ideal, I have tried using the fly proxy which also seemed like it might be a solution, but had no luck with this also.

11

In that case, I’d suggest using the .internal address instead…

Alter the binding to [::]:8090 (without the 1), since otherwise it’s only reachable from within that same Machine.

Then, after making sure that the Machine is still running, access as…

http://my-app.internal:8090/

(These .internal addresses are generally simpler than Flycast, since they bypass the Fly Proxy completely. There is no SSL handler, port remapping, or auto-start, for example.)

And if that still fails, we can fall back to fly proxy—which is actually what I’m more familiar with myself.

2 Likes
12

If you want to use flycast, you can allocate a private ipv6 address via fly ips allocate-v6 --private

13

I am connected to my existing wireguard setup, I have altered the binding to [::]:8090 I can ping6 _api.internal -c4 and get a response.
But http://my-app.internal:8090/ cannot resolve the host.

I also tried http://my-app.flycast:8090 while connected to through wireguard and it tries a little longer (so appears to resolve the host) but ultimately the connection is reset by peer, so maybe the bind is wrong maybe at my application if nothing is listening at the private IP I created?

14

Hm… Maybe try fly dig my-app.internal, which will check from a different, easier direction.

(The my-app.internal should really only fail to resolve if the Machine has stopped.)

Success will look like…

;; ANSWER SECTION:
my-app.internal. 5  IN  AAAA  fdaa:7:651:a7b:f:1234:fedc:2

(This is just the name-resolution part, and not trying to connect to any ports yet.)

The Flycast address would require you to add a [[services]] block to your fly.toml, since that’s how the Fly Proxy knows which ports you want it to assist with. However, doing so might inadvertently expose your admin panel to the public.

15

Huzzah! It works on http://my-app.internal:8090 !

I setup the services in my fly.toml and connected to my wireguard tunnel. Thanks @mayailurus for you help. I will endeavor to write this up in another thread with an example app in another thread.

I suppose by virtue of having a vpn tunnel I am secure and would need to issue my own https cert in order to use https.

1 Like
16

Glad to hear it!

(These private networks are a favorite feature of mine, as it turns out…)

It would be prudent to double-check that your admin panel isn’t also being exposed to the public on http://my-app.fly.dev:8090 or https://my-app.fly.dev:8090—I think.

Some configurations mixing port 8090 and [[services]] would do so by accident.

(That’s why I would nudge a bit more toward .internal-only here.)

Overall, it shouldn’t be necessary to have any mentions of port 8090 in fly.toml, if you’re not using the .flycast address.

You’re right, though, that your own traffic over the VPN is opaque to prying eyes.

I mentioned SSL in passing above only because my own reflex is to add https: in front of every curl invocation, these days.

⋮

Aside2: I hear that some organizations do use SSL on top of WireGuard, in the “mutual TLS” form. That’s not for the encryption, as I understand it, but rather for the mutual authentication.

17

The other main reason you’d want to use TLS on top of WireGuard is when you want to use browser APIs that are limited to only work over TLS/HTTPS. Not to mention avoiding that “Not secure” warning. I’m gonna go see if it’s possible to have the platform manage TLS certs for you when using Flycast.

1 Like
18

I doubled checked that https://my-app.fly.dev:8090 is not available. .internal works well, I think there was a few pieces of the puzzle I was missing. I think having TLS/HTTPS would be a good addition even when using wireguard.

1 Like