CPU autoscaling

1

Correct me if I’m wrong, but AFAIK autoscaling in Fly only works based on concurrent connections.

Is CPU usage also taken into account?

Eg:

I’m working on a service for encoding audio files with ffmpeg which is pretty heavy CPU intensive. Ideally I’d want a new VM to spin up if the current VM is already encoding a file.

2

We don’t currently take that into account no. We might in the future as we are improving our balancing algorithm.

Eventually we might even be able to let people define exactly which metrics are relevant for balancing and scaling their app.

1 Like
3

Hooking up to a Fly-hosted prometheus metric would be great! This is how I have achieved scaling based on a Buildkite queue depth using the Kubernetes HPA, with much pain involved :smiley:

4

What if I set up concurrency to 1?

  [services.concurrency]
    hard_limit = 1
    soft_limit = 1

Would that spin up a new VM on every new concurrent request?

5

We don’t scale fast enough to do that effectively. Right now, scaling happens as a background process that uses metrics to pick the count. Metrics are only scraped every 15s, so there’s some lag before we even start a VM.

1 Like