Hi,
replaying requests to an app or specific instance in fly.io is great, but there is some room in between. Sometimes, we need more control over the load balancing method, which selects the instance that the request is routed to, but we don’t want to handle the routing in our code (possibly by building a list of instances and selecting a specific one).
In my use case, I need to route certain clients to the same instance, but still balance the load. My naive proposal would be a Kafka (the message broker) like functionality. Messages are put on different partitions of a topic, to which producers and consumers then attach. The producer doesn’t need to know the number of partitions and their names, but provides a key which is then used to ensure, that all messages with the same key end up on the same partition in order, by using some kind of modulo hashing. I imagine something like this would be easy to implement in the load balancing algorithm of the proxy, by providing a routing key.
If possible, the hashing should be stable when adding new instances or removing instances.
Any thoughts on this?