kill_timeout not working as expected

[jivanovic]

Hello Fly Community I have 2 app instances with performace-2x machines running. I configured kill_timeout inside of the fly.toml but noticed that it does not respect the value I set when downscaling an instance. I used a kill_timeout value of 300 (5 minutes), however once the downgrading signal is sent the instance shuts down instantly. Should I also set a kill_signal value?

Here is my fly.toml file:

app = "..."
primary_region = "iad"
kill_timeout = 300

[build]

[http_service]
  internal_port = 3000
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0
  processes = ["app"]

Here is a screenshot of the shutdown process logs. As you can see the instance is shut down as soon as the signal is sent.

[roadmr]

Hi there!

The log seems to indicate your kill_timeout change didn’t get applied - the interval between SIGINT (signal the application to shut down gracefully) and SIGTERM (ok, you didn’t shut down gracefully, now just go away) is 5 seconds, which is the default.

If you can get your app to respond to SIGINT and shut down gracefully though, that’s definitely the best option kill_timeout is there in case your app’s shutdown takes more than 5 seconds.

Did you fly deploy when you changed the kill_timeout configuration? You can double-check the effective configuration with fly config show.

• Daniel

[jivanovic]

Hey @roadmr Thank you for the reply, I really appreciate it!

I would prefer to use kill_timeout since it is somewhat specific to my use case. You can read more about it in this thread Instance Downscaling

I checked the fly config show and it looks like kill_timeout should be applied, however I am still testing it and it does not seem to be working… Here is the fly config show:

Do you have any further ideas on why that could be?

One more thing, the Billing Dashboard is constantly loading for me today and I cannot get the anticipated monthly charge. I checked the statuses and everything seems to be working on your part but the Billing Dashboard never seems to load fully

[MatthewIngwersen]

Hey @jivanovic—after looking into it further, we discovered a backend bug where the kill_timeout was being ignored when a machine was fully stopped. The patch is being rolled out, so hopefully this will work for you now.

Thanks for the report, and if this doesn’t seem to fix the issue for you, please let us know!

(The dashboard issue should be resolved now too.)

[jivanovic]

Hello @MatthewIngwersen The kill_timeout is applied as expected! Thank you for the fast patch, you guys are the best

The dashboard issue is also resolved