Issues deploying a simple app

I can successfully deploy this app GitHub - tilleryd/fly-simple by running flyctl launch, but If I change anything, even just the text in the body of the HTML, future deployments get hung up at the pushing image stage.

$ flyctl deploy
Deploying dawn-sound-7275
==> Validating app configuration
--> Validating app configuration done
Services
TCP 80/443 ⇢ 8080
==> Creating build context
--> Creating build context done
==> Building image with Docker
--> docker host: 20.10.8 linux x86_64
Sending build context to Docker daemon  73.22kB
[+] Building 0.5s (8/8) FINISHED                                                                                                                                                                            
 => [internal] load remote build context                                                                                                                                                               0.0s
 => copy /context /                                                                                                                                                                                    0.1s
 => [internal] load metadata for docker.io/library/node:16                                                                                                                                             0.4s
 => [1/4] FROM docker.io/library/node:16@sha256:580a0850049c59a48f06090edd48c9f966c5e6572bbbabc369ba3ecbc4855dba                                                                                       0.0s
 => CACHED [2/4] WORKDIR /usr/src/app                                                                                                                                                                  0.0s
 => CACHED [3/4] COPY ./build .                                                                                                                                                                        0.0s
 => CACHED [4/4] RUN npm install -g serve                                                                                                                                                              0.0s
 => exporting to image                                                                                                                                                                                 0.0s
 => => exporting layers                                                                                                                                                                                0.0s
 => => writing image sha256:07315aafd1349a9e040f3cdce704d4c53763fdc170f983e6c602128bf105fee0                                                                                                           0.0s
 => => naming to registry.fly.io/dawn-sound-7275:deployment-1638331952                                                                                                                                 0.0s
--> Building image done
==> Pushing image to fly
The push refers to repository [registry.fly.io/dawn-sound-7275]
75f64f64b507: Pushed 
9aba3a654ada: Pushed 
d82c83a5fba4: Layer already exists 
0ddf3378b186: Layer already exists 
110d3632d653: Layer already exists 
a46c3bae9c2e: Layer already exists 
b485db677a87: Layer already exists 
6cdb6a376c8b: Layer already exists 
a5e5ff41e5d3: Layer already exists 
0ac0b7cace95: Layer already exists 
ee64586bec22: Layer already exists 
040a44cff5e0: Layer already exists 
deployment-1638331952: digest: sha256:822c27ae000124056469fdbbb07b45af4cda5960fd03c0202c15b152932fece1 size: 2839
--> Pushing image done

I’ve been exploring fly.io today and I keep running into this problem. A new app will launch without issues but then any change to it fails to deploy. I feel like I’m missing something really simple. Does anyone know what is causing this? Thanks.

This is unusual. Can you try with fly deploy --remote-only to run the deploy from one of our remote builder VMs?

Also, you can try LOG_LEVEL=debug fly deploy to see where it might be getting stuck with standard deployment.

Running with fly deploy --remote-only worked perfectly. I did run with LOG_LEVEL=debug but that didn’t provide any smoking gun as far as I can tell. Below is the snippet showing the attempt to push the image to fly. The error appears after cancelling the process and it might imply that the image never successfully gets pushed, hence the failure to delete it.

...
==> Pushing image to fly
The push refers to repository [registry.fly.io/dawn-sound-7275]
75f64f64b507: Layer already exists 
9aba3a654ada: Layer already exists 
d82c83a5fba4: Layer already exists 
0ddf3378b186: Layer already exists 
110d3632d653: Layer already exists 
a46c3bae9c2e: Layer already exists 
b485db677a87: Layer already exists 
6cdb6a376c8b: Layer already exists 
a5e5ff41e5d3: Layer already exists 
0ac0b7cace95: Layer already exists 
ee64586bec22: Layer already exists 
040a44cff5e0: Layer already exists 
deployment-1638367135: digest: sha256:822c27ae000124056469fdbbb07b45af4cda5960fd03c0202c15b152932fece1 size: 2839
--> Pushing image done
^CDEBUG Error deleting imageDelete "http://%2Fvar%2Frun%2Fdocker.sock/v1.41/images/registry.fly.io/dawn-sound-7275:deployment-1638367135": context canceled
DEBUG result image:&{ID:sha256:07315aafd1349a9e040f3cdce704d4c53763fdc170f983e6c602128bf105fee0 Tag:registry.fly.io/dawn-sound-7275:deployment-1638367135 Size:913693090} error:<nil>
Image: registry.fly.io/dawn-sound-7275:deployment-1638367135
Image size: 914 MB
==> Creating release

Why does this work but not locally?

What you see happening there after cancelling the build is an error from your local machine. After pushing the image, flyctl will delete it from your local Docker installation to save space. For some reason, your local Docker is not allowing this. It might be worth giving it a kick/restart.