Autoscale doesn't seem to launch new instances

There’s not yet! We’re redoing our orchestration layer so we can boot new VMs without having to do metrics queries. The lag is not great, we’d like to be able to spin one up immediately and hand a request off to it. This is a big project, though, so it’ll take a few months to get there.

We’d also like to enable scaling with things like CPU, but those suffer from the same lag. It might be 15-30s before we “see” a CPU spike. In practice the metrics based scaling works ok, but as you found from stress testing, it’s not as responsive as it should be.

2 Likes