Autoscaling is not triggered on a pure websocket application

Ah, I think I understand what you are looking for now (instances to scale up in the specific region where soft limit has been reached). Indeed, the existing documentation did describe this kind of region-aware behavior:

  • Standard: Instances of the application, up to the minimum count, are evenly distributed among the regions in the pool. They are not relocated in response to traffic. New instances are added where there is demand, up to the maximum count.

I dug into this recently, and found that our current (nomad-based) autoscaler implementation was deployed around Sept 2021, but this documentation was originally written for an earlier system that had somewhat different behavior. Most notably, there’s only one ‘mode’ in the current autoscaling system (and I believe the scaling action is a simple adjustment of the total instance count), so setting ‘balanced’ mode no longer has any effect.

Apologies for the confusion our quite out-of-date documentation has caused on this. I’ve prepared some updates to the documentation and flyctl that should clarify the current behavior a bit better moving forward.

That said, it does look like the current autoscaler is not quite capable of what you’re looking for. Our current efforts have been focused on adding features to Machines to help manage more advanced scaling requirements like this. Autoscaling apps on machines isn’t quite ready yet, but it’s something we’re actively working on and should eventually be able to better handle your use-case.

1 Like

Ok, I think we will disable autoscaling for launch and revisit the topic later then. We don’t have too much traffic for now so we can totally live with a fixed set of VMs in specific spots around the globe (the main goal being optimal latency).

However I’m not sure how to do that. I have this set of regions configured:

Region Pool:
cdg
hkg
lax
maa
scl
syd
Backup Region:
ams
sjc

How can I start a VM in each of these regions? I tried to disable autoscaling and to set the VM count to 6 with flyctl scale count 6, hoping it would fill the spots, but instead it started 4 in lax and hkg:

Instances
ID      	PROCESS	VERSION	REGION	DESIRED	STATUS 	HEALTH CHECKS     	RESTARTS	CREATED
6a3dd886	app    	950    	hkg   	run    	pending	                  	0       	8s ago              	
10cbf6c9	app    	950    	lax   	run    	pending	                  	0       	8s ago              	
2c3a79e6	app    	950    	lax   	run    	pending	                  	0       	8s ago              	
dd84e7c3	app    	950    	hkg   	run    	pending	                  	0       	8s ago              	
d24f1a6d	app    	950    	scl   	run    	running	2 total, 2 passing	0       	2h24m ago           	
784474bd	app    	950    	cdg   	run    	running	2 total, 2 passing	0       	2022-10-21T13:20:11Z	

I also tried switching back to autoscaling with a min count of 6 but got the same result.

Fly’s main selling point is, for us, starting VMs close to users, how can I do that? :slight_smile:

The --max-per-region option to flyctl scale count is probably what you’re looking for. For example flyctl scale count 6 --max-per-region 1 should start a single VM in each of 6 different regions.

2 Likes

Thanks that should work for us. Do you have plans for the autoscaling that would better fit our needs? Is there an issue, a changelog, or a Twitter account I can track to keep updated on this subject?