Unable deploy new versions

I am unable to deploy new versions. After running fly deploy it just gets stuck in the health check stage. fly logs don’t reveal any useful info. However the last line was

2022-05-10T23:13:56.088 app[01901d11] maa [info] Reaped child process with pid: 619, exit code: 0

Try running fly vm status 72f99efe and see if it says anything about the health check failing. This seems like what would happen if the health check doesn’t pass in time.

Here’s the output:

Instance
  ID            = 72f99efe    
  Process       =             
  Version       = 24          
  Region        = maa         
  Desired       = run         
  Status        = pending     
  Health Checks =             
  Restarts      = 0           
  Created       = 56m12s ago  

Recent Events
TIMESTAMP TYPE MESSAGE 

Checks
ID SERVICE STATE OUTPUT 

Recent Logs

I am having this issues randomly today too. I woke up to two GitHub action jobs taking 6 hours then being cancelled (which annoyingly used up a chunk of my minutes :sweat:).

The build doesn’t get pass this step

Run flyctl deploy --remote-only --app [my-app]
6 ==> Verifying app config
7 --> Verified app config
8 ==> Building image
9 Waiting for remote builder fly-builder-bitter-silence-192...
10 Remote builder fly-builder-bitter-silence-192 ready
11 ==> Creating build context
12 --> Creating build context done
13 ==> Building image with Docker
14 --> docker host: 20.10.12 linux x86_64

I have rerun the build a few times this morning and it continues to get stalled here. I stop it after it attempts for 5mins on this step. My region I am attempting to deploy to is syd.

Failed due to unhealthy allocations in syd - #6 by joshua this might be related @charklewis

1 Like

@charklewis can you try running without --remote-only? Github actions can be a little brittle with remote builds, we’re doing some reasonably sophisticated network stuff that conflicts with the GH environment sometimes.

@kurt this worked! It did take over 6mins though (normally my build/deploy takes ~1min). Would there be a reason for it randomly failing last night with --remote-only?

Is it safe to continue using this in my actions?

The network setup GitHub actions requires can fail if something between them and us is slow. Unfortunately, we can’t cache that between runs (like we do on your local host).

--remote-only is just brittle on GitHub Actions, it’s hard to tell exactly why it fails at any given time.

1 Like

Gotcha, no worries. I’ll keep on eye on it in the future and it starts failing again I will try without --remote-only. If it does start failing again, should I flag it here? Happy to help try to debug this in the future if it is helpful todo so.

@kurt just a heads up it is actually all of my deployments (across a few apps) failing in Github Actions when using --remote-only. Not a major issue as I can remove this and it works. I will update here when they start working again.

I am doing local deploy though.

Ok, now that I am checking, it has been deployed. Took a lot of time though.