WireGuard only access to server with custom domain?

Questions / Help
1

Greetings;

Is it possible for a fly app with no services to use a custom domain and be visible over WireGuard?

I had an application that had two ports exposed and I updated my fly configuration to remove them. My expectation was I would be able to visit the site while on WireGuard. Unfortunately, I encounter errors when trying to reach the site from a browser (while using WireGuard). The Internal port is 8080

  1. https://.internal:8080 - does not resolve
  2. https://:80 - PR_END_OF_FILE_ERROR

I am trying to determine if it is feasible to have a custom domain that only people in the organization can see while on WireGuard. How is this done today?

Kind Regards

2 Likes
2

I’m not totally sure I follow what you’re doing here, but:

  • The [[services]] in your fly.toml instruct our Anycast network how to route traffic; it doesn’t impact what’s exposed for private networks.

  • The .internal domain addresses private networks only. So ports you have open on your Fly.io VMs should all be available on your .internal domain, regardless of what [[services]] says in your fly.toml.

  • Desktops, dev machines, and servers running outside of Fly.io need to have a WireGuard peer set up to reach .internal names (ie: any name you would see with flyctl dig). Moreover: they’ll need to be pointing to the DNS server in the WireGuard configuration (not always super convenient to get this working on desktop OSs).

3

@thomas thanks for circling back to my question.

To circle back to my use case: I would like to know if the following is possible from a development machine:

  1. Visit https://<custom-domain>.internal/... and it resolves the address then sends traffic to the server process.
  2. Visit https://https://<custom-domain>/... and it does not resolve for those not on the private network.
  3. Each device would be a member of the organization to avoid #2.

I understood zero services section meant no public internet for a given service. I got this understanding from the App Configuration [documentation](https://fly.io/docs/reference/configuration/#the-services-sections).

No services section: The application has no mappings to the external internet - typically apps like databases that talk over 6PN private networking to other apps.

I am pretty sure there is documentation putting all this together, but I can not seem to find what the appropriate configuration is to accomplish my goal.

For example, I tried the following command:

 flyctl dig AAAA <custom-domain>.internal:<$INTERNALPORT> --app ... -o ...

And received:

;; QUESTION SECTION:
;<custom-domain>.club.internal:$PORT.	IN	 AAAA

;; AUTHORITY SECTION:
.	2475	IN	SOA	a.root-servers.net. nstld.verisign-grs.com. 2022012600 1800 900 604800 86400

For reference when I run the same dig command for the custom domain I get:

 flyctl dig AAAA <custom-domain> --app ... -o ...
;; opcode: QUERY, status: NOERROR, id: 52569
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;<custom-domain>.	IN	 AAAA

;; ANSWER SECTION:
<custom-domain>.	900	IN	CNAME	$FLY_APP_DOMAIN.
$FLY_APP_DOMAIN.	3600	IN	AAAA	.....

I hope this is a greater context to understand what I am trying to accomplish.

Moreover: they’ll need to be pointing to the DNS server in the WireGuard configuration (not always super convenient to get this working on desktop OSs).

I am running on Mac OS and yeah this is not pleasant. I did not update my DNS entry at the system level prior to this conversation, but I have that setup now. Unfortunately after updating the settings a ping to the internal host/port does not resolve.

4

@thomas I thought a bit more about your question and I also did some research and I think I have a different way of stating my problem.

Is it possible to use a custom domain and it resolves .internal addresses? It appears if it is the case there may be more machinery than adding a CNAME/TXT record.

Thanks for any feedback.

5

I see what you’re asking. That won’t work, I don’t think. You can CNAME a public DNS name to myapp.internal, that’s true. But when a browser resolves the CNAME, it’s still left needing to resolve myapp.internal to get the AAAA address, and the only way to reliably do that is for the browser to hit our nameservers. We’re happy to have your browser do that, but unfortunately, Windows and macOS aren’t so happy about it.

6

Hi @thomas,

Can you help me understand what you mean by these please.
I have a vague interpretation of it and my below comment is based off of that.
Please do correct me if it’s wrong.

Moreover: they’ll need to be pointing to the DNS server in the WireGuard configuration (not always super convenient to get this working on desktop OSs).

We’re happy to have your browser do that, but unfortunately, Windows and macOS aren’t so happy about it.

… to get the AAAA address, and the only way to reliably do that is for the browser to hit our nameservers.

Do you mean point it to fdaa::3, as described here?

OP can create a new Firefox profile (if OP can use Firefox), and either
a) set DoH for the profile to Fly’s DNS (does Fly’s DNS support DoH?)
b) fiddle with dns settings in about:config and override system defaults

7

We do not do DoH on our internal DNS, nor do we have any plans to (it doesn’t make sense in our security model, where all traffic is already cryptographically secure).

8

Ok, well, I mentioned DoH because in the Firefox network config, you can point to DNS (if it’s via DoH). So just wondering if creating a new FF profile with DNS to Fly will work for OP.

Anyway, I suppose use case like this will be possible soon thanks to per-container* VPN, i.e., Protect your container tabs with Mozilla VPN | Mozilla VPN Help.

*Firefox browser-tab container

Currently, it’s plumbing between the Mozilla VPN (which is WireGuard based) extension and Firefox containers. So these tab containers can already proxy http(s), socks, etc.
wg userland per tab container, if/when it happens, will be ideal.

9

Yeah, it’s not a bad thought. Just be aware that your desktop OS probably doesn’t want to use our DNS server for routine stuff (same goes for your browser), so getting .internal names to resolve in random browsers is going to be painful. If that’s the kind of thing you’re looking for, I’d recommend putting a Tailscale agent in the apps you run here. Solving these kinds of problems is basically their full-time job.