Autoscale from metrics

Currently the autoscaler is based on the number of connections.

Do you have any plans to support alternative means of autoscaling? It would be awesome if there were options to scale horizontally (more instances) and vertically (more CPU/MEM).

For data ingestion workloads this would be super helpful and make them more elastic. Something along the lines of Kubernetes pods autoscaling with Kafka metrics | by Roman Noze | Engineering at Palo Alto Networks | Medium

1 Like

We vaguely plan to bake this in. The current autoscaler is actually not what most people want, so we’re starting by tackling that. What they usually want is for machines to start on demand when there are a certain number of connections.

Scaling with different strategies is something I think people might end up building with the machines API. You can run a machine that wakes up every so often, and then starts other machines for you. Or run one that continuously polls metrics and starts and stops other things.

1 Like

I experimented with a Machine that didn’t run a server, but just a process (ie, no [[services]]). I don’t remember being able to start it with the fly m start command. It was tore down everytime. But if I understand you right, this usecase is supported (but it doesn’t / didn’t work…)?

Autoscale, believe it or not, is/was one of the most interesting among Fly’s features which sadly isn’t getting the eng love it deserves.

In fact, the Fly proxy needs to observe and react in-time to (an app’s / a process’ / a machine’s) increasing / decreasing requests / connections wrt soft_limit and hard_limits, which in my experience, it doesn’t quite do so. I imagine it is a hard problem (and relates to autoscaling in some sense) no doubt… but one that needs solving.

@kurt glad to hear improvements are on the radar.

@ignoramous very much agree here. It feels super close to being what people naturally expect + adaptable to other contexts like stream processing.

Autoscale, believe it or not, is/was one of the most interesting among Fly’s features which sadly isn’t getting the eng love it deserves.