Issues with machines restart policy

[catflydotio]

@pacsni It sounds like you’re still having the issue you reported in, e.g. , Guide: Elixir App v1 to v2 migration (It was easy) - #16 by pacsni?

i.e. you’re trying fly machine update with definite config changes, and getting back Error no config changes found instead of a diff and a request to confirm?

Can you confirm that with the latest stable flyctl, running fly m update --restart always <machine-id> doesn’t work?

[pacsni]

@catflydotio

fly m update --restart always

Hey,

I am on 0.0.504 .
When I run this command I see:

flyctl machine update --restart always 91857d7f2d6983 -a APPNAME
Searching for image 'registry.fly.io/APPNAME:deployment-01GX2NH37D1554RQN3P4BWVXEJ@sha256:7c86850b8bb252ca95e4a20f7fa9af541f3fba351e662c9e4926216f50537747' remotely...
image found: img_dozgpek1e6mp096y
Image: registry.fly.io/APPNAME:deployment-01GX2NH37D1554RQN3P4BWVXEJ
Image size: 151 MB

Error no config changes found

For that other report, the scale command worked.

[catflydotio]

Thank you.

[tj1]

What are the different restart policies? I couldn’t find the appropriate documentation.

[catflydotio]

There are three policies (using the names of the options in fly m update and fly m run help):

no - always let a Machine stop when its main process exits, whether that’s on purpose or on a crash

always - never let a Machine enter the stopped state, even if your main process exits cleanly

on-fail - try up to 10 times to restart the Machine if it exits with a non-zero exit code, before letting it stop. This is the default behaviour if the policy is not specified.

Right now, the default on fly machine run is always. Fly Postgres apps are configured with a policy of always, too.

The default for a new Fly App (V2 of course, since V1 doesn’t use Machines) is an empty policy, equivalent to on-fail. This lets Machines be restarted if they crash, and allows your app Machines to effectively scale down by exiting cleanly, but:

@JP_Phillips

The exit event indicates the VM shutdown on its own with exit code 0. It’s possible this machine was on a host that rebooted and we currently can’t easily detect if a machine stopped due to the host shutdown signal or some other reason. If the machine is meant to always be running, it’s a good practice to set a restart policy of “always” so flyd will restart the machine regardless of exit code.

At the moment:

If Fly Proxy can wake it on request (regular web app Machine with services configured), or you do active scaling via the API, the default on-fail policy should be good.

For an always-on app with no services: host reboots could stop Machines. If you want to set and forget, always is the best policy.

Right now, fly machine update (or the Machines API) is the way to change the policy. It doesn’t exist in app-wide configuration yet, so fly.toml won’t help.

[tj1]

Is there a way to disable wake on request?

[pier]

Sorry for the confusion it will cause but I edited the title of this thread.

[catflydotio]

@tj1

Is there a way to disable wake on request?

I believe the answer to this one is “not yet but soon.” As far as I’m aware, today, if your Machines have services defined, Fly Proxy will wake them up when it happens to hit them in its load-balancing efforts.

[tj1]

Would you know if removing machines out of the pool on the roadmap somewhere?