Having an issue deploying an Elixir BE service

Questions / Help
1

Hey, I’m having an issue deploying to fly.io. I deployed it successfully a few days ago and nothing changes in fly.io configs, now I’m getting this.

Error: failed to fetch an image or build from source: error building: input:3: ensureDepotRemoteBuilder {"code"=>"internal", "message"=>"internal error"}

Is there anything I can investigate?

Thank you!

2 Likes
2

Yep, same thing here…

1 Like
3

Ditto. And https://status.flyio.net/ shows no errors yet. Sigh. I guess we wait…

1 Like
4

Same here, all deployments started to fail.

1 Like
6

Thanks for the reports, we’re looking into the issue with remote builders. In the meantime, you can try using --local-only to build locally as a workaround, or --depot=false to use a fly-hosted builder.

7

(base) XXX@XXX-MacBook-Pro-2 frontend % flyctl deploy --remote-only --strategy bluegreen --no-cache --local-only
==> Verifying app config
Validating /Users/XXX/listingai-v2/frontend/fly.toml
✓ Configuration is valid
→ Verified app config
==> Building image
==> Building image
Error: failed to fetch an image or build from source: docker is unavailable to build the deployment image

Any other suggestions?

8

Same here!

9

All working already

10

Should we expect some post-mortem or this is BAU for fly.io? :slight_smile:

11

remove --local-only from your invocation - the last option (either remote or local) wins, and since in this case “local wins”, you don’t have Docker installed locally and the local build can’t proceed.

So either:

  • --local-only (but you must have Docker installed)
  • --remote-only --depot=false (use a non-depot remote builder, doesn’t need local Docker installed)
  • Daniel
12

Now works fine without any changes from my side, thanks

13

I see you joined 18 minutes ago, this is BAU…they might write up something, but the root issue here is obviously going undressed as these kinds of issues (and much worse) continue to happen on a weekly basis. In fact, this is one of the less impactful issues because our existing deployments were still accessible during the outage.

14

Thanks, yeah i got it going with --depot=false

16

Yes… You might try putting a watch on the postmortem tag; typically they come out the week after the incident—after they’ve had a chance to collate their own records, discuss internally what did actually happen, explain what measures are being taken to fix the underlying causes…

(“Watch” is the little bell icon.)

Aside: According to the status page, it was a “a database outage” at Depot, the third party that runs the default builders. But there is generally a lot more detail in the full, subsequent write-up.

2 Likes
17

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.