Is it possible to make scaling more deterministic?

Questions / Help
1

I’ve read the docs and all the nice ways to do scaling. However, amidst all the options I find myself still missing what I’d really prefer. What I need (for example) is a way to tell fly to run 3 instances of my app in this fra, and 5 in ewr, thats it, just a fixed setting. Let me explain why all the scaling options do not work for me:

Count Based Scaling
This doesn’t work for me because it evenly distributes all my instances across all my regions. However, I have more users in some regions so I’d like to tip the scales that way. My traffic is not super bursty so I’d like to just set some numbers in those regions. I had actually been using count scaling and just have the same number of servers in all my regions, but I just had a deployment “fail” in an interesting way. After a deployment, I had my 2 app instances sent to one region as opposed to evenly spreading it out. This incited a bit of panic as I scrambled to figure out why I lost service in one geo. Deploys should be deterministic, I should not feel like Im rolling the dice whenever I do a deploy. I think what happened here is that during the deployment it chose one geo and a backup region in that geo, and completely knee-capped my other geo.

Anchor Scaling
This is supposed to be the de-facto option if you want to specifically set instance counts in specific regions. However, I find it to be a very scary and confusing option for a live app. For instance I once tried deleting the volume to go back to count based scaling and my entire service went down (granted this was also my fault for deleting a live mounted volume). Also I had count based scaling, so I had lets say 6 instances of my app running in 3 geos. As soon as I pinned the app to the first volume I lost ALL the other apps running in other geos. It wasn’t obvious if there was a way to preserve my apps in all regions while I enabled this option. Also its not exactly clear how this option deals with backup regions. What if I pin to a region that is not able to bring up my server at deploy time but I have no volumes in the backup regions?
It also feels odd to have to attach a volume in order to specify scaling options, why can’t I just tell fly where to run the instances?

Autoscaling
This option seems fine, but again I personally prefer to just provision the capacity I need in regions I care about. Again, here the baseline is that all instances are evenly distributed across all regions in my list. Which I do not want.

I hope some of the explanations above help. I think I am good for now (except for the fact that count based scaling did the wrong thing when I last deployed), however I’d appreciate some comments to help me understand why there isn’t an option to just say where to deploy instances and how many.

2 Likes
2

Following!

I too would enjoy a somewhat simpler set of static & platform-guaranteed {region, count, vmtype} knobs. It’s how we’re used to managing our scale policy in previous environments (heroku, ecs), so it would make any migration to fly simpler too.

With such an option, the “we will auto place instances according to our conditions” policy would just be an alternative, default policy/scaling regime; appropriate for some apps but not necessarily others.

2 Likes
3

The most fine-grained scaling is coming to Fly in a few months.

It’s still a WIP product, but it already powers our remote builders. It lets you decide on the specs and region of every VM.

We’re making progress on this, but it’s not yet done.

If you’d like to try it out, pull the latest flyctl and type fly machine. It’s rough around the edges and you can’t expect too much support for it yet :slight_smile:

3 Likes
4

Hey, that’s cool news! Will definitely give it a spin, cheers.

5

That would we awesome @jerome! One more thing though could you at least help explain the situation with my count-based scaling not working? Is there a bug there perhaps? I had count based scaling set up for 6 instances with my region list as ewr,fra. However after deploying an updated I ended up with all 6 instances in fra,cdg. Is this expected?

6

What does fly regions list show?

There are no guaranteed minimum number of instances in each region. Instances might spill over to backups regions too.

If you set backup regions the same as your primary regions, then you won’t have any backups regions, that might be more desirable.

1 Like
7

Perfect! thanks, might be worth adding this bit of info to the docs. Stuff like this would greatly help folks like myself understand how scaling works and remove the “fear” or worry.

Thanks again! I still love fly like a fat kid loves cake!