@gernotstarke I’m not seeing the same issue for your organization. Would you mind running again after setting LOG_LEVEL=debug so we can see the details of where it’s failing?
I’m also still getting the same error after:
- Destroying all apps with “fly apps destroy <app_name>”
- Confirming the list is empty with “fly apps list”
- Deleting the fly.toml file
- Launching new app with “fly launch”
- Deploy new app with “fly deploy”
Output is
==> Verifying app config
Validating /<path>/fly.toml
Platform: machines
✓ Configuration is valid
--> Verified app config
==> Building image
WARN Failed to start remote builder heartbeat: server returned a non-200 status code: 500
Error: failed to fetch an image or build from source: server returned a non-200 status code: 500
1 Like
Thank you for the report @Tassadar! We are seeing where this is failing but have not yet determined why. Will post an update when we have one.
This is also my first time using fly.io (literally installed it about 2 hours ago) so I assumed it was user error until I stumbled upon this thread. Very unlucky timing for me!
Been getting the same error for around 24 hours at this point, tried destroying the builder twice - once yesterday and a second time just now - still same error.
(I’m using a Dockerfile and my app is being deployed via CD with Github if this is relevant, although when I try to deploy locally from my PC I get the same error.)
Run flyctl deploy --remote-only
==> Verifying app config
--> Verified app config
Validating /home/runner/work/<app>/<app>/fly.toml
Platform: machines
✓ Configuration is valid
==> Building image
WARN Failed to start remote builder heartbeat: server returned a non-200 status code: 500
Error: failed to fetch an image or build from source: error connecting to docker: server returned a non-200 status code: 500
Error: Process completed with exit code 1.
Edit:
30ish minutes after I posted this I managed to deploy. Didn't change anything, it just worked when I tried again. Thank you fly team!
I’m in the exact same position.
When I run fly apps list it seems the running machine is in a constant status of pending and is unusable.
Hi all—we discovered that a recent change was inadvertently causing volume creation for remote builders to fail for users without a payment method on file. We rolled out a fix a few minutes ago.
We’re still looking into a few potential issues with builders, but hopefully this unblocks some of you! Thanks for your patience.
2 Likes
That unblocked me! Thank you! I destroyed the app and redeployed, all is well on my side.
I was experiencing the same error in Phoenix, AZ.
Destroyed app and redeployed:
1. Destroyed app with “fly apps destroy <app_name>”
2. Confirming the app was no longer on the apps list with “fly apps list”
3. Deleted the fly.toml file
4. Launched new app with “fly launch”
5. Deployed new app with “fly deploy”
Issue resolved.
Note: addressing the issue with this route will require you to reset any secrets. Possibly other settings will need to be reconfigured. I would have liked to not have to do this, and it is possible that fly has resolved the issue for other users in a different way. I was not able to wait any longer and rebuilding resolved the issue!
I had a similar error
resolve image config for docker.io/docker/dockerfile:1:
that would resolve with a 503 timeout.
I took the recommendation of this thread and destroyed the builder. Now I am getting
WARN Failed to start remote builder heartbeat: server returned a non-200 status code: 504
Error: failed to fetch an image or build from source: error connecting to docker: remote builder app unavailable
like everyone else. My new builder periodically says
2023-10-14T03:30:14.060 runner[48ed1e0a754708] iad [info] Pulling container image docker-hub-mirror.fly.io/flyio/rchab:sha-a4467b8
2 Likes
I’m experiencing a similar issue right now. It started with a GH action failure 20 hours ago. I tried destroying the builder app and restarting in several different regions, but got the same Error: failed to get image: error connecting to docker: server returned a non-200 status code: 504 error each time. Most recently, I tried destroying and recreating my own app, but that resulted in the same error. I have a valid credit card linked to my account.
Same issue here:
Platform: machines
✓ Configuration is valid
--> Verified app config
==> Building image
WARN Failed to start remote builder heartbeat: server returned a non-200 status code: 504
WARN Remote builder did not start in time. Check remote builder logs with `flyctl logs -a fly-builder-bold-wood-2870`
Error: failed to fetch an image or build from source: error connecting to docker: remote builder app unavailable
Having the same issue:
WARN Failed to start remote builder heartbeat: server returned a non-200 status code: 504
Same exact issue as the posters immediately before me. 504 error. FRA region.
Same here 504 on all our services in bothAMS and FRA regions…
Same issue, DFW region.
Deployed, deleted, launched, deployed. Still failed.
Error: failed to fetch an image or build from source: error connecting to docker: remote builder app unavailable
Same here, ruby, hkg.
flyctl logs -a builder-app
2023-10-14T06:51:25Z runner[1111] hkg [info]Pulling container image docker-hub-mirror.fly.io/flyio/rchab:sha-2222
2023-10-14T06:52:50Z runner[1111] hkg [info]Pulling container image docker-hub-mirror.fly.io/flyio/rchab:sha-2222
2023-10-14T06:54:15Z runner[1111] hkg [info]Pulling container image docker-hub-mirror.fly.io/flyio/rchab:sha-2222
2023-10-14T06:55:40Z runner[1111] hkg [info]Pulling container image docker-hub-mirror.fly.io/flyio/rchab:sha-2222
2023-10-14T06:57:05Z runner[1111] hkg [info]Pulling container image docker-hub-mirror.fly.io/flyio/rchab:sha-2222
2023-10-14T06:58:30Z runner[1111] hkg [info]Pulling container image docker-hub-mirror.fly.io/flyio/rchab:sha-2222
2023-10-14T06:59:56Z runner[1111] hkg [info]Pulling container image docker-hub-mirror.fly.io/flyio/rchab:sha-2222
i have same issue like this