Failed deployment, it reverted back but the app took too long to come back up

So today I’ve tried to deploy a new image from Github action. It took forever to run the release command (rails db:migrate in my case). I cancelled that, run the same deployment command from terminal, same problem. Inspecting the app logs with fly logs showing that something took forever at Unpacking image.

I thought something might be wrong with the release, so ran the deployment for the web server app. It failed with error unhealthy allocation, and proceeded to revert back. Then I got 525 from Cloudflare trying to access my web server. Apparently, the app couldn’t book up, something seems stuck for a long time, until about 15 minutes later it went back online.

Was there any incident that could cause this problem? I just feel like doing deployment on fly is a bit of a risk, and don’t feel confident anymore.

So another problem with deployment, now it’s stuck forever at Running release task (pending).... There was a topic created a year ago for this same issue that was marked as resolved, but I couldn’t find any meaningful resolution there…

Can I get some attention for this deployment issue please?

Another update on this matter: now the deployment gone through, but getting the same SSL handshake error as in above picture… I’m too afraid of deployment now

It looks like your app has a PRIMARY_REGION env var set to syd, but that’s not one of the regions you have set for it. We run release commands in the PRIMARY_REGION, so they’re hanging.

Is this intentional? Try setting fly regions set syd and I believe the deploys will work properly.

Thanks for your reply. Interesting. Yes for the Bookforward-stg app, I did set the env and then set it to sin manually with fly region set. Let me remove the env var then.

What about the other deployment issue where the app couldn’t start up and get the ssl handshake error? Do you have any advice? I get this problem on production app, not the Bookforward-stg one,