@thomas can give you more details here, but:
Our ipv6 is basically an overlay network, just a very simple one. Each organization gets it’s own network ipv6 prefix. This is one of my wireguard peer IPs:
fdaa:0:863:a7b:bea:0:a:2/120
The fdaa:0:863:: is my prefix, we only route traffic from other fdaa:0:863:: IPs. fdaa:0:864:: is an entirely different, isolated network.
The actual network this runs across is a wireguard mesh. The “tricks” are all related to routing IPv6. We have a different IP space we use for this, it just so happens to share components.
:a7b:bea:
This uniquely identifies a physical host in our infra (kind of like a router, I guess). We route all of fdaa:a7b:bea:: across wireguard to the same host – it’s one of the AllowedIPs in our core wireguard config. We do this by rewriting the 6pn address:
fdaa:0:863:a7b:bea:0:a:2 -> fdaa:a7b:bea:0:863:0:a:2
Then do the reverse when it hits the host router.
It’s amazingly simple, which is good! There’s very little that can go wrong with our specific logic. We don’t route customer 6pn networks at all, and the rewritten version is always different than 6pn prefixes. The most “brittle” part is restricting what gets routed to which VM, but it’s (again) pretty simple because we can just match prefixes.