Setting a minimum number of instances to keep running when using auto start/stop

Fresh Produce
1

We now support setting a minimum number of machines to keep running when using the automatic start/stop feature for Apps v2. This will prevent the specified number of machines from being stopped. Update your flyctl to the latest version and then in your fly.toml

[[services]]
  auto_start_machines = true
  auto_stop_machines = true
  min_machines_running = 1
  ...
  ...

Similarly if you’re using http_service

[http_service]
  auto_start_machines = true
  auto_stop_machines = true
  min_machines_running = 1
  ...
  ...

When should you use this?*

If instances of your application take a while to start and that is unacceptable for your use case, you will benefit having at least 1 instance always running (min_machines_running = 1). When a new request comes in, instead of having to wait for the app to start up in the case it was scaled down completely (i.e the cold start problem), it is able to respond immediately.

What you need to know

The most important is that we only keep instances running in the primary region of your app. All other regions will still get scaled down to 0. As an example, if min_machines_running = 3, then you’ll need 3+ instances in your primary region.

Some other things to know:

  • The max number of machines we can scale up to is implicitly defined by the number of machines your app has. We will scale your app all the way up if the demand requires it and scale back down to the minimum specified
  • The default minimum is set to 0

* This does not solve the cold start problem entirely. When a request comes in and the proxy decides to start a new instance, that request waits for the new instance to start. We don’t start a new instance while servicing the current request with an already running instance. So while you may not run into a cold start for your first instance, if we start a second one, that request will run into it. We’re giving some thought to how to solve this and as always, will post on here once we’ve got a solution for you :smile:

16 Likes
2

Awesome @senyo ! Thanks so much for tackling this feature so fast. :rocket:

I just tried it out, but I have the feeling, that it doesn’t work correctly.

My fly.toml looks like this:

app = "peter-kuhmann-website"
primary_region = "ams"

[http_service]
  internal_port = 3000
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 1

I have two machines (I cloned one of them just recently).

According to the monitoring and the logs, both machines got scaled down:

2023-05-12T19:05:56.552 proxy [6e82d956a79408] ams [info] Downscaling app peter-kuhmann-website in region ams. Automatically stopping machine 6e82d956a79408. 2 instances are running, 0 are at soft limit, we only need 1 running

2023-05-12T19:05:56.558 app[6e82d956a79408] ams [info] Sending signal SIGINT to main child process w/ PID 513

2023-05-12T19:05:56.746 app[6e82d956a79408] ams [info] Starting clean up.

2023-05-12T19:05:57.746 app[6e82d956a79408] ams [info] [ 405.553727] reboot: Restarting system

2023-05-12T19:07:18.119 proxy [5683d920b1618e] ams [info] Downscaling app peter-kuhmann-website in region ams. Automatically stopping machine 5683d920b1618e. 1 instance is running but has no load

2023-05-12T19:07:18.122 app[5683d920b1618e] ams [info] Sending signal SIGINT to main child process w/ PID 513

2023-05-12T19:07:18.628 app[5683d920b1618e] ams [info] Starting clean up.

2023-05-12T19:07:19.630 app[5683d920b1618e] ams [info] [ 503.747677] reboot: Restarting system

Interesting: 2 instances are running, 0 are at soft limit, we only need 1 running on first downscale it seems to “know” the min setting.

But the second check doesn’t seem to take it into account: 1 instance is running but has no load.

Did I miss a specific configuration or precondition?

Best
Peter :slight_smile:

1 Like
3

Appreciate the kind words!

Ah, it’s possibly due to an outdated flyctl version, what version are you using? I forgot to mention to upgrade it in the post, I’ve added it in now.

1 Like
4

That was the solution. Now it works as expected. Brilliant!

Learned something as a Fly-newbie: New feature, check for flyctl updates! :smiling_face:

4 Likes
5

Awesome, glad it’s working!

6

Superb! A lot of great work by a really great team!

1 Like
7

This is awesome! I literally made a post about this pain point a few days ago and came to the forums for another reason to see that its implemented as a feature!

Thanks for all the hard work fly team :slight_smile:

2 Likes