Hey @irano ,
I know what you mean. Fly Proxy is not very smart when it comes to handling load and distributing it across multiple machines. Sometimes in my apps, one machine is throttling while the other one is up and healthy, yet the traffic is NOT balanced between them and keeps hitting the one that is struggling.
They suggested setting a hard_limit, but if there is more than one machine running, “”“I don’t care”“” about hard limits — traffic should be balanced evenly anyway (in my opinion, we are paying for both machines, not just the one that is busy).
My workaround has been to run multiple processes to handle different workloads and place an Nginx reverse proxy in front to distribute the load, which adds complexity to the infrastructure.
FYI: I also reported this issue a few weeks back, but I don’t think anything has changed since my previous threads. Still, it’s “good” to know that someone else is experiencing the same problem.
I hope the Fly.io team adds this to their backlog.
Past threads:
- Fly-ifying our internal DNS - #2 by Luan_Oliveira
- Fly Proxy routing most traffic to a single machine with low concurrency CPU-bound requests
PS: This text was originally written in Portuguese and translated into English using AI.