I heave e.g. 3 regions (ewr, lax and lhr).
According to the user usages (if the metrics data is reliable) I’m getting like 10-20x more users in ewr.
All the autoscaling features etc. seems to not work at all…
So I rather prefer to distribute the instances manually. E.g. I wanted to have like 3 instances in ewr and 1 in lax and lhr.
It seems there was a CLI command that did exactly that in a very simple way:
flyctl scale regions ams=1 hkg=1 sjc=1 from Scaling Fly for All · Fly )
This is now deprecated…
I tried to mimic that doing the following:
- First, I only set the ewr region setting
- Then I scale (with regular
scale, not the
autoscale) 3 instances… The are all set to ewr
- Then, I only add lax, and scale to 4, lax i added
- Then, I add lhr, and scale to 5… I’m hoping to found the 5th instance running in lhr… But nope…
I tried going back and forth with
scale count 4 and
scale count 5, and checking with
status if that 5th instance is running in lhr, but it’s always running in lax…
I also tried with
scale count 6 after having done 5 with two instances in lax, and hopefully that 6th one was indeed in lhr…
But TBH, this is so ridiculous…
Does anyone know, how I can manually set the instances per region?
Genuine question to the fly team, why did you deprecated the above CLI command?
I could have simply done
flyctl scale regions ewr=3 lax=1 lhr=1.
It’s a safe bet that the people at fly are working on better autoscaling.
The dream would be to have instances magically start up where the traffic is coming from as the traffic is growing, and wind down as the traffic winds down with natural day/night/traffic cycles. It’s just the next logical step after “scale to zero”.
If you want database locality maybe you could also whitelist the regions.
Ideally it will improve to a point where manually setting instance scaling per region would only be needed if you have very unique circumstances.
+1 Came here to troubleshoot a similar situation where I have 10x users in FRA and no way to load balance requests or scale a specific region. This means load average is through the roof while 5 idle instances stand by
I can maybe move traffic with multiple Fly apps + a Cloudflare load balancer or Worker, but that is not ideal. Wonder if there’s a better way?
Here’s my mental model / what I’ve been looking for. I think it’s slightly in conflict with how Fly wants me to think of regions.
For completeness & by comparison, I think it logically works more like this today (all regions form a single pool - vs having single region pools):
Have you tried balanced autoscaling?
Balanced: Instances of the application are, at first, evenly distributed among the regions in the pool up to the minimum count. Where traffic is high in a particular region, new instances will be created there and then, when the maximum count of instances has been used, instances will be moved from other regions to that region. This movement of instances is designed to balance the supply of compute power with demand for it.
I haven’t tried it myself, just curious to know.
Balanced autoscaling hasn’t really worked well for me. I believe it’s scaled by concurrent requests, which doesn’t always map to load. I haven’t been able to get it to spin up more instances in the correct region when needed.