Is there any way to balance the load between machines depending on the pool of connections to the database? Let’s say I have 4 VMs, and each VM has a limit of 40. Is it possible to somehow transfer users to the machine with the fewest connections?
I’m not an expert on Fly built-in networking, but you can always experiment with a traditional proxy, like HAProxy or Traefik. I assume they generally containerise well (and I had a good experience with Traefik some years back).
In this approach, your four application servers would need to stream connection count data to the LB, so the LB can prioritise the machine with the lowest load. I assume that it would not matter if the connection count was slightly out of date, so perhaps the VMs could send a count every ten seconds or so.