Deployment release_command timeout issue

[dwhite5514]

I’ve been hitting an issue today deploying to our production environment using the superfly/flyctl-actions/setup-flyctl action and a command like:

flyctl deploy --remote-only -c fly-prod.toml

When it get’s to running our release_command, I’m hitting this:

-------
 ✖ Failed: error running release_command machine: timeout reached waiting for machine's state to change
-------
Error: release command failed - aborting deployment. error running release_command machine: timeout reached waiting for machine's state to change
Your machine never reached the state "stopped".

You can try increasing the timeout with the --wait-timeout flag

Our release_command in our fly-prod.toml is:

release_command = '/app/bin/migrate'

When I look at the temp machine created to run the release_command it seems it’s always in the destroyed state. At least I can’t tell if it started and died right away or what. When I try to look at the logs for the temp machine, all I see are two log entries saying it’s pulling and then successfully prepared our image. Nothing else.

I have successfully deployed today the staging version of our app which is in another org/project so I’m at a loss for what’s going on with the prod deploy. Finally I tried the same “flyctl deploy …” command from my dev system (not GH action) and got the same error. It looks like it’s hitting the 5 minute timeout for the release_command as mentioned in the docs here: App configuration (fly.toml) · Fly Docs

Any ideas what’s going on?

[mayailurus]

Hm… I would try fly console --image registry.fly.io/<long-string>, citing the particular image that the failing release command was created from. This will give you the same kind of ephemeral Machine it was using, to experiment inside of, etc.

(This is fly console without the ssh in the middle, by the way.)

Also, it’s wise to start fly logs in a separate terminal window and leave it running there the entire time. The historical logs aren’t always complete, unfortunately.

Hope this helps a little!