I am unable to deploy new versions. After running
fly deploy it just gets stuck in the health check stage.
fly logs don’t reveal any useful info. However the last line was
2022-05-10T23:13:56.088 app[01901d11] maa [info] Reaped child process with pid: 619, exit code: 0
fly vm status 72f99efe and see if it says anything about the health check failing. This seems like what would happen if the health check doesn’t pass in time.
Here’s the output:
ID = 72f99efe
Version = 24
Region = maa
Desired = run
Status = pending
Health Checks =
Restarts = 0
Created = 56m12s ago
TIMESTAMP TYPE MESSAGE
ID SERVICE STATE OUTPUT
I am having this issues randomly today too. I woke up to two GitHub action jobs taking 6 hours then being cancelled (which annoyingly used up a chunk of my minutes ).
The build doesn’t get pass this step
Run flyctl deploy --remote-only --app [my-app]
6 ==> Verifying app config
7 --> Verified app config
8 ==> Building image
9 Waiting for remote builder fly-builder-bitter-silence-192...
10 Remote builder fly-builder-bitter-silence-192 ready
11 ==> Creating build context
12 --> Creating build context done
13 ==> Building image with Docker
14 --> docker host: 20.10.12 linux x86_64
I have rerun the build a few times this morning and it continues to get stalled here. I stop it after it attempts for 5mins on this step. My region I am attempting to deploy to is syd.
@charklewis can you try running without
--remote-only? Github actions can be a little brittle with remote builds, we’re doing some reasonably sophisticated network stuff that conflicts with the GH environment sometimes.
@kurt this worked! It did take over 6mins though (normally my build/deploy takes ~1min). Would there be a reason for it randomly failing last night with
Is it safe to continue using this in my actions?
The network setup GitHub actions requires can fail if something between them and us is slow. Unfortunately, we can’t cache that between runs (like we do on your local host).
--remote-only is just brittle on GitHub Actions, it’s hard to tell exactly why it fails at any given time.
Gotcha, no worries. I’ll keep on eye on it in the future and it starts failing again I will try without
--remote-only. If it does start failing again, should I flag it here? Happy to help try to debug this in the future if it is helpful todo so.
@kurt just a heads up it is actually all of my deployments (across a few apps) failing in Github Actions when using
--remote-only. Not a major issue as I can remove this and it works. I will update here when they start working again.
I am doing local deploy though.
Ok, now that I am checking, it has been deployed. Took a lot of time though.