Unable to deploy: Waiting for depot builder...

Questions / Help
,
1

Hey all, I’ve been unable for the last hour or so. Deploys stuck at “Waiting for depot builder…”

This seems to be a recurring issue, is there anything in flight to fix this a little more permanently?

Using --depot=false is also failing for me:

==> Verifying app config
Validating /Users/robgough/code/stayupfront.com/web/fly.toml
✓ Configuration is valid
--> Verified app config
==> Building image
Remote builder fly-builder-black-rain-4153 ready
WARN Failed to start remote builder heartbeat: parse "http://fdaa:39:21e7:a7b:4a3:92c5:774a:2:8080/flyio/v1/extendDeadline": invalid port ":39:21e7:a7b:4a3:92c5:774a:2:8080" after host

==> Building image
✓ compatible remote builder found
INFO Override builder host with: https://fly-builder-black-rain-4153.fly.dev (was tcp://[fdaa:39:21e7:a7b:4a3:92c5:774a:2]:2375)

Remote builder fly-builder-black-rain-4153 ready
INFO Override builder host with: https://fly-builder-black-rain-4153.fly.dev (was tcp://[fdaa:39:21e7:a7b:4a3:92c5:774a:2]:2375)

Remote builder fly-builder-black-rain-4153 ready
==> Building image with Docker
--> docker host: 24.0.7 linux x86_64
[+] Building 0.0s (0/0)
Error: failed to fetch an image or build from source: error building: failed to get status: rpc error: code = Unavailable desc = rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: unable to upgrade to h2c, received 500"
1 Like
2

Hey @robgough

Have you tried the -–local-only flag? It works for me when I’m facing issues with depot.

3

I’ve not actually, as I’m running on a Mac traditionally I’ve only used --remote-only though I can’t remember why I originally started doing that many moons ago.

I should probably give it a go, but this is quite a big change I’m about to deploy and probably not a good one to try it on, but will make sure I give it a go later!

Thanks

1 Like
4

What I would try now is: Org > Settings > App Builders > Reset

In fact, I only use the local-only when facing issues with depot as following:

flyctl deploy --local-only --ha=false --config ./fly.prod.toml

1 Like
5

Ah, going there I was surprised to see my builder was in DFW which is currently having issues – thought it would be in LHR or at least Europe. Don’t think I’ve seen that settings page before for the builders.

Just kicked off a deploy after changing that, and forgetting to specify --depot=false and it now appears to have sprung to life again.

Thanks for the responses!

1 Like
6

I’m also experiencing this never-ending “Waiting for depot builder…” I tried switching my App Builder from DFW => ORD but this doesn’t seem to have had an effect.

7

@kmozzler sorry for the trouble — could you try another region like IAD or SJC?

8

Thanks @kyle . I was able to get one of my apps to deploy using that region. I have another non-critical app that is running in ORD and has a volume in ORD. It’s hosed. It’s not running. Won’t spin up. I’m unfamiliar with how to migrate the volume and machine to a different region, or if I even can do that. It’s not a huge deal to wait it out and see if starts working again, but I also wouldn’t mind knowing how to move it should this happen to an app that I don’t really want down

9

Sorry about that, I think I see the app you’re talking about. Can you try a volume fork/machine clone to another region for now? We have a good guide on this: Using Fly Volume forks for faster startup times · Fly Docs

If you have trouble with that I can reach out via DM

10

Thanks for that tip - resetting it worked for me - I’m in ORD.

1 Like
12

Resetting builders for me in DFW didn’t help, stuck on waiting for depot

13

Could you try change your builder region? Resetting effectively just brings up a new builder Machine in the same region, but DFW is seeing intermittent start failures so brining up a builder in a new region should work.

14

I went to Org > Settings > App Builders > Configure and changed the region from DFW to IAD. The build started immediately.

1 Like
15

Can confirm switching the builder to ORD fixed it for now

16

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.