Connect on-premise networks to Fly via Wireguard

[jcrapuchettes]

Fly offers a Wireguard VPN usually used from an individual’s computer. However, there are situations where having access to the VPN from an entire private network is beneficial. This can be difficult because Fly doesn’t offer a /64 subnet, a requirement for IPv6 Router Advertisement. The solution to this is NAT66 (Network Address Translation for IPv6-to-IPv6). Basically, we enable the router to connect to Fly through a WG peer and then advertise an IPv6 network internally that will be NAT’d through the Fly IPv6 address. Without the NAT, there isn’t a route for the return traffic to the internal network. This guide will layout how to get the connection working with both a pfSense router and a Mikrotik router.

pfSense Setup

Wireguard Configuration

1. Install WG package under System → Package Manager
2. In a console, run fly wg create and fill in the appropriate information
3. Under the VPN tab, select WireGuard
4. Add a new tunnel
   1. Add a useful description (e.g. fly)
   2. Pick a listening port
   3. Paste into the Private Key the private key from step 2
   4. For interface address, paste in the address from step 2
   5. Click Save Tunnel
5. Add Peer
   1. Select appropriate tunnel
   2. Add a useful description (e.g. fly-sea)
   3. Unselect Dynamic
   4. Paste endpoint from step 2
   5. Enter value from step 2 for Keep Alive
   6. Paste paste public key from step 2
   7. Leave Pre-shared Key blank
   8. Fill in Allowed IPs from step 2
   9. Click Save Peer
6. Under Interfaces tab, select Assignments
   1. Add the new fly tun_wg* network port
   2. Click on the new interface name
   3. Change Description to something more descriptive (e.g. Fly)
   4. Under Static IPv6 Configuration, make sure the address is correct based on step 2’s information
   5. Add a new gateway
      1. Gateway IPv6 address, should be the interface address defined in step 2
   6. Click Save
7. Last we need to create a static route for all traffic going to our Fly /48
   1. Go to System → Routing → Static Routes
   2. Click Add
   3. For Destination network, fill in the AllowedIPs from step 2
   4. Set Gateway to the new fly v6 gateway
   5. Click Save

At this point the router should be able to ping the IP of any Fly node.

DNS Configuration

So that we can communicate with all of the nodes in the Fly org by name, we need to forward DNS requests for the internal TLD to Fly’s DNS server. These steps assume that you are using the DNS Resolver, but they might work for the DNS Forwarder.

1. Services → DNS Resolver
2. Scroll to the bottom of the page and click Add under Domain Overrides
3. Set Domain to internal
4. IP Address should be the address of the DNS server reported when the WG connection was created
5. Set Description to “Forward .internal to fly”
6. Click Save

At this point, you should be able to use Diagnostics → DNS Lookup to lookup the address(es) of one of your apps by .internal.

NAT66 Configuration

This is the last major configuration that needs to be made to enable the internal network to communicate to Fly through the router. We create a private IPv6 subnet on the LAN, add a NAT rule, and then advertise the subnet.

1. First, we need to define a /64 for the internal network to use and to be routed to Fly.
   1. Go to Interfaces → LAN (or whichever interface you want to be allowed to connect to Fly)
   2. For the IPv6 Configuration Type, set to Static IPv6
   3. Scroll down and set the IPv6 address to something in the fd00::0/8 range with a /64 subnet.
   4. Click Save
2. Go to Firewall → NAT
   1. Go to Outbound
   2. Select Hybrid Outbound NAT
   3. Click Save
   4. Click Add to add an outbound NAT rule
   5. Select the appropriate interface
   6. Address Family: IPv6
   7. Source: Network and set the network equal to the IPv6 range defined in 1.c above
   8. Click Save
3. Last, we need to advertise that the defined IPv6 address can be used for routing.
   1. Go to Services → DHCPv6 Server & RA
   2. Select the appropriate interface tab
   3. Select the Router Advertisements tab
   4. Change the Router mode to Unmanaged
   5. Click Save

And you are done!

Mikrotik Setup

For the Mikrotik setup, I’m primarily a WinBox user so all configuration will be with respect to that user interface.

Wireguard Configuration

1. In a console, run fly wg create and fill in the appropriate information
2. In WinBox, click WireGuard on the side bar and then the + to add a new interface
3. Give the interface a name
4. Click the down arrow to the right of the Private Key and paste in the private key from step 2.
5. Click OK to save
6. Go to the Peers tab and click + to add a new peer
7. Select the appropriate interface that was just created
8. Paste in the Public Key, Endpoint, and Endpoint Port
9. For the Allowed Address, fill in the CIDR block specified from step 2 as AllowedIPs
10. For Persistent Keepalive, step 2 specifies the number of seconds
11. Click OK

IPv6 Addresses and Private Subnet

1. On the side bar, click IPv6 → Addresses
2. Click + to add the fly peer address
3. Set the Address to the Address specified in the wireguard configuration (step 2 above)
4. Set the Interface to the new wireguard interface
5. Uncheck the Advertise checkbox
6. Click OK to save
7. Now add another address for the subnet by clicking the +
8. For Address, set to something in the fd00::0/8 range with a /64 subnet.
9. Set the Interface to the bridge
   NOTE: This adds the CIDR block to your LAN network. You might need to change this if your router/switch is configured differently.
10. Make sure that the Advertise checkbox is checked.
    Note: This advertises to the network that the router is able to route for this address. Other computers in the network will now allocate their own addresses within that CIDR block via SLAAC.
11. Click OK
12. Now we need to create a static route for your Org’s Fly network by going to the side bar and clicking IPv6 → Routes
13. Add a route with the + button
14. Dst. Address should be the AllowedIPs from the wireguard configuration (step 2 in first section)
15. Gateway should be set to the new fly wg interface
16. Click OK
17. Lastly, we need to add the NAT66 rule to send traffic to Fly through the WG tunnel. On the side bar, click IPv6 → Firewall.
18. Go to the NAT tab
19. Create a new NAT rule with the following setting:
    • Chain: srcnat
    • Dst. Address should be the AllowedIPs CIDR block from the wireguard configuration (step 2 in first section)
    • Action: src nat
    • To Address: Address from the wireguard configuration (step 2 in first section)
20. Click OK

DNS Configuration

So that we can communicate with all of the nodes in the Fly org by name, we need to forward DNS requests for the internal TLD to Fly’s DNS server. These steps assume that you are using the DNS server on the Mikrotik for your network.

1. On the side bar, click IP → DNS
2. Click the Static button
3. Add a new static entry with the + button
4. Name is optional
5. Fill in Regexp as .*\.internal$
6. Change the Type to FWD
7. Paste into the Forward To field the DNS address from the wireguard configuration (step 2 in the first section)
8. Click OK

And you are done! Enjoy your easy access to your Org’s Fly private network and DNS.