Machine doesn't start on deployment

I have two apps with a scale count of 1 running in fra as primary and ams as a secondary region. They’re configured with bluegreen deployment strategy.

Recently when I’m deploying an update the green machines would get created, and then only the primary machine in fra would start. The deploy process would just sit in perpetuity until timeout in created state (like below). The only way I can get around this is to run fly m start <machine_id> after which it immediately starts and completes the deployment process.

Creating green machines
  Created machine 148e276f3ee448 [app]
  Created machine 4d89629a404168 [app]

Waiting for all green machines to start
  Machine 148e276f3ee448 [app] - created
  Machine 4d89629a404168 [app] - started

Can someone from Fly advise on why this is happening?

What version of flyctl do you have installed?

There was a recent update where autostart/autostop apps don’t start stopped machines during deployments. That update may have impacted you.

flyctl v0.1.120 linux/amd64 Commit: 437e9fa9cb1a3046953f9f5a88cdc5784352f18a BuildDate: 2023-11-09T15:59:20Z

flyctl cli autoupdates now no?

A little confused. In a deployment shouldn’t all affected machines start regardless of the runtime scale configurations? Unsure whether you’re suggesting this is the current behavior with flyctl or whether it should have been resolved already?

In version 0.1.119 the logic was changed so that stopped machines wouldn’t need to be started for them to be updated. This is extremely useful when deploying an app that has 10s or 100s of machines with many of them stopped.

It seems that maybe there is a bug when the deployment configuration is bluegreen instead of rolling.

You could try reverting to version 0.1.118 for now to get around this. I’d also suggest filing a bug report in the flyctl github repo Issues · superfly/flyctl (github.com) and mention this thread.

2 Likes

Sorry about that, this PR should fix it. - Fix: stopped autoscaled machines in bluegreen deployments by wjordan · Pull Request #3019 · superfly/flyctl · GitHub

I’ll release a new version shortly.

1 Like

The fix is released in flyctl version 0.1.121 - run fly version upgrade to upgrade.

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.