Machine Left Running despite auto stop and no traffic

I have deployed several apps with the same application (auto start/stop enabled) , with an additional file of varying size to investigate if this affects cold starts.

I sent requests every 15 minutes, always generating a cold start.
However, since 24.09 , two of these identical apps have had a machine permanently started.

Now, even after stopping all traffic to the apps, they are still started and won’t automatically scale down.

In the logs:
App has excess capacity, autostopping machine . 1 out of 2 machines left running (region=arn, process group=app)

I never changed the configuration. Why is this happening?

I do see this happening on a few of your apps. I initially thought you might have min_machines_running = 1 set on these apps—but that’s not the case.

I see they have not been deployed in a couple of months or more, though. Can you update flyctl and re-deploy the apps to see if it fixes the issue? Might be tied to this regression:

@luccadibe

This should be fixed now. Machine state got desynchronized between flyd (our VM runner) and service discovery system that the proxy uses on a single host where two of your apps had a machine.

1 Like

It’s fixed! That was quick, thanks.

I have been tracking the cold start latency (recording TTFB) for more than month, and noticed that it got considerably worse over time for some of the apps, and then it reset back on Sept 10th.
The distribution is very similar for some of them.
It may be of interest to you:

1 Like

Added autoscaling, proxy

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.