Error connecting to docker: remote builder app unavailable

Questions / Help Build debugging
1

Hi, yesterday I deployed just fine with fly deploy.
Today there is this error after some minutes trying to contact builder.

WARN Remote builder did not start in time. Check remote builder logs with `flyctl logs -a fly-builder-still-morning-9002`
Error failed to fetch an image or build from source: error connecting to docker: remote builder app unavailable

I tried deleting the builder and creating a new one in the dashboard, but the result is the same.

I also tried removing and recreating a wireguard peer connections, enabling wireguard websockets and other suggestions posted in other relevant topics.

What steps should I take now?

2

I have the same issue. Is this an issue from fly.io?

3

Same here. Every deploy is timing out for me today.

==> Verifying app config
--> Verified app config
==> Building image
WARN Remote builder did not start in time. Check remote builder logs with `flyctl logs -a fly-builder-sparkling-violet-5498`
Error failed to fetch an image or build from source: remote builder app unavailable

Iā€™ve tried deleting the builder, so it automatically creates a new one on next deploy attempt, but it still fails to deploy with the new builder.

I also tried scaling the builder up (are we allowed to do that??) but no luck.

My builder and the app Iā€™m trying to deploy are both in the lhr region (London, UK).

4

I wanted to add that I can run fly ssh console and other fly commands successfully on the running app.
(Even start the observer (!) and observe the app remotely b/c itā€™s a Elixir/Phoenix app)

Only the command fly deploy fails for me.

5

@gtaranti can you try fly apps destroy fly-builder-still-morning-9002 and retry?

@callum same, but with the app fly-builder-sparkling-violet-5498.

@hmontazeri I donā€™t know what your builder app name is, but if you can try and destroy it and then retry a deploy, that might help.

1 Like
6

@Jerome Thanks, but the same with new builder : fly-builder-fragrant-smoke-1646

7

Same, I have tried destroying the builder. Just tried again now too. Deploy always fails.

When I monitor the builderā€™s logs, it always gets to Configuring firecracker and then nothing.

Also, not sure if this is related, but: my postgresdb app (also in lhr region) has just died after I attempted to scale it, and it last log line is Configuring firecracker, same as on the builderā€™s logs. The monitoring page says: ā€˜Failed deployment v7 ā€¦ Failed due to unhealthy allocations. ā€¦ 1 desired, 1 placed, 0 healthy, 1 unhealthy"ā€™

8

Thanks @callum for letting us know the region!
@hmontazeri and @gtaranti what region(s) are yā€™all deploying in?

9

Iā€™m trying to deploy an app in region fra
For the builder I donā€™t know. Does it matter which region is the builder?

10

The logs show the builder not booting correctly.

Iā€™ve create an incident as weā€™re working on resolving this: Fly.io Status - Remote builders are not working properly in London

11

Can all of you try again now? I think itā€™ll work better.

12

FRA

==> Building image with Docker
--> docker host: 20.10.12 linux x86_64
[+] Building 597.2s (0/1)                                                       
 => [internal] load remote build context                                 597.2s
Error failed to fetch an image or build from source: error building: error during connect: Post "http://[fdaa:0:6775:a7b:23c4:1:183b:2]:2375/v1.41/build?buildargs=null&buildid=upload-request%3Abcd8662ef86ccf001c4ccbd4ea35255276a06ee47c87a41e4ee85f9782557f56&cachefrom=null&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=&labels=null&memory=0&memswap=0&networkmode=&rm=0&shmsize=0&target=&ulimits=null&version=2": write unix ->/Users/montazha/.fly/fly-agent.sock: write: broken pipe
13

Only after destroying the builder and trying again fly deploy , I was able to deploy successfully.

Issue resolved for me.

14

Thanks, I am able to deploy again now. And my postgres db app is back online.

15

Hi there!
I have the same problem on github action. Thanksfully, most deployments success. But sometimes, they dont and I get this error. Any idea of what the reason could be?

**;5;252m--> Verified app config
==> Building image
Waiting for remote builder fly-builder-shy-wave-9903...
Error failed to fetch an image or build from source: error connecting to docker: remote builder app unavailable


WARN Remote builder did not start in time. Check remote builder logs with `flyctl logs -a fly-builder-shy-wave-9903`
Error: Process completed with exit code 1.

Thanks :wink:

16

I was having the same issue and found a solution.
The problem for me was that while i donā€™t have a paid plan, i was deploying on a region that is ā€œPaid Plan Onlyā€ (fra).
The issue was resolved when i changed that.