Let’s suppose I have an app with 4 machines in 4 regions.
I would like to create “groups” of machines related to each other. For example, group A would contain machine 1 and machine 2, and group B would contain machine 3 and machine 4
I have three needs:
A machine from a group must be able to find the IPv6 address of other machines of the same group.
A machine from a group must be able to connect to another group – ie to connect to the closest machine of the target group.
A public user must connect to the closest machine of the closest group via a single URL.
Here, a “sub-app” is a group of machines with a proxy to allow connection to the nearest machine in the group.
Both sub-apps need to communicate to each other via the proxy (in the example, machine #2 would connect to machine #3 instead of machine #4, because #3 is closer).
I’ve read all about Private Networking, and I almost can achieve that… but not quite.
I’m thinking about two potential solutions:
Create a fly app for every group of machines. They can now communicate easily with each other. But then how to create a “super-app” that proxy those multiple sub-apps? A random user must be able to access the nearest machine from one single public url.
Add a way to define “groups” of machines, and the ability to get the internal IPv6 address of the closest machine of a group. Something like that: top1.nearest.of.<app>.<group>.internal. That would allow a machine from group A to connect to the closest machine from group B.
It appears that top999.nearest.of.<app>.internal will return an ordered list.
There is an API to set metadata for a machine: Dynamic machine metadata; and <value>.<key>.kv._metadata.<appname>.internal can be used to query it.
Assuming #1 holds AND the total number of machines is not humongous, with two DNS queries and a set intersection you can get a list of machines in an arbitrary subgroup ordered by distance.
It sounds like you have a plan that works for you, so I don’t want to unsettle that, but after seeing this:
I just wanted to add that the DNS already groups machines by region, so you can query <region>.<appname>.internal, and furthermore if you want to get machines which belong to a specific Fly App process group in a specific region you can query <process_group>.process.<region>.<appname>.internal, so I think you could do this without needing to set any custom metadata at all.
Again, don’t let me knock you off your horse though. If you see your way forward, then keep going, the metadata kv method will also work.
You’re not knocking me off my horse, you’re trying to find a better stallion
Indeed, we could call <region>.<appname>.internal for every region that we know belong to the group. We receive several IP addresses, and we find the closest with the topX.nearest call.
That would work at the cost of additional DNS requests (+1 DNS request per unique region in the group). But these requests are only done once, so that’s not a big deal.
