I cannot deploy

I am trying to deploy a Remix Js application but I cannot. Some of the errors are listed below:

failed to fetch an image or build from source: error fetching docker server info: Get "http://[fdaa:0:4453:a7b:1449:0:77d9:2]:2375/v1.24/info": context deadline exceede

ERRO[0606] Can't add file /home/geees/kazungu/projects/platform/sisters_school/build/_assets/tailwind-VPYM37M7.css to tar: io: read/write on closed pipe 
ERRO[0606] Can't close tar writer: io: read/write on closed pipe 
Error failed to fetch an image or build from source: error building: unexpected EO

Does this error happen again if you retry?

I have just tried to deploy and I am getting the following error


Waiting for remote builder fly-builder-morning-forest-1101... 
connecting ⡿ Error error connecting to docker: unable to
 connect WireGuard tunnel: context deadline exceeded

That happens a lot for me too.

What I do is destroy the builder and then try another deploy.

And if that fails with the same error, wait a bit, destroy the builder, and try again. Eventually it works.

fly destroy builder-app-name

2 Likes

I will try this and see what is going to happen.Thank you.

No problem, it’s the only problem I have right now.

Super annoying when you want to deploy, like I do now, and it waits and waits and waits … then fails. Wastes a load of time waiting, deleting, retrying etc.

Hopefully there is a fix incoming.

Hi, we’ve been unable to deploy for a few days. Generally we build via our CI but deployments kept failing with:

Error failed to fetch an image or build from source: error rendering push status stream: name unknown: app repository not found

I tried again by running fly deploy manually and got up to…

 => [build  6/17] RUN mix deps.get --only prod &&   mix deps.compile     519.3s
 => => # remote: Total 249 (delta 55), reused 66 (delta 26), pack-reused 124   
 => => # * Getting phoenix_live_view (https://github.com/phoenixframework/phoen
 => => # ix_live_view.git)                                                     
 => => # remote: Enumerating objects: 17332, done.                             
 => => # remote: Counting objects:  61% (2475/4056)      

…but it got stuck here for about 10 minutes before I killed it.

Next I tried destroying our builder and ran manually again and this happened:

[...]
==> Verifying app config
--> Verified app config
==> Building image
Waiting for remote builder fly-builder-withered-fog-8661... connecting ⣷ Error failed to fetch an image or build from source: error connecting to docker: unable to connect WireGuard tunnel: context deadline exceeded

Any idea what’s happening? We haven’t changed anything on our end. Our last successful deployment was last Friday (Jan. 14).

Not sure about the first half of your post, that error, but regarding the second issue, I’ve diagnosed with @kurt : it seems that time-out is caused by a lack of a config.yml file.

If you run a deploy locally, that is present from any prior successful builds. You should see it in your home folder, in a .fly folder. Its got a load of wireguard stuff in.

But if you run a deploy from CI, that config.yml does not persist from prior builds and so it is very slow and can time out before all the details are gathered (as i understand it).

See:

(so don’t destroy the builder. I was doing that and found it helped but I now think that was just a coincidence that it appeared to be helping, and actually was unconnected to the underlying config yml issue)

Thanks, unfortunately after adding --local-only to the fly deploy command in my CI config I’m still getting the first error…

==> Pushing image to fly
The push refers to repository [registry.fly.io/production]
c15ee4d375c1: Preparing
8dd379b875dc: Preparing
bb84c9222e03: Preparing
9a18678f8624: Preparing
d761cd9de8e5: Preparing
72e830a4dff5: Preparing
72e830a4dff5: Waiting
Error failed to fetch an image or build from source: error rendering push status stream: name unknown: app repository not found

I have tried your suggestion and still can’t deploy.

Error

Waiting for remote builder fly-builder-morning-forest-110
1... connecting ⣾ Error error connecting to docker: 
unable to connect WireGuard tunnel: context deadline exceeded

Is deployment working ?

Deployments are working for me.

I’m not sure why @enaia is getting that other error as I’ve not come across that issue, with repository not found. Think that is one for Fly.

But as regards the:

... unable to connect WireGuard tunnel: context deadline exceeded

error, which I used to get a lot and tried e.g deleting the builder and other random stuff, I’ve solved it by making sure there is a config.yml file present. If there isn’t, I get that error. As it waits several minutes and gives up. If there is, it proceeds immediately to deploy.

So look in the output of the deploy command, a bit further up. Do you see a line about a missing config.yml? It should show where it is expecting it to be found.

It seems like that config.yml is made on a successful deploy, so if you don’t have one, it may be you need to deploy until you get one, and then keep it. That is made much harder with CI builds as generally the server is made fresh each time, losing prior files, and hence this happens way more times. I persisted the config.yml to solve that:

… the other option is not using a remote builder at all, bypassing the issue. That requires you to have docker installed. If you do, great, change --remote-only to --local-only, and that will use your local docker, and so again, avoid the connection to the wireguard tunnel etc.

It seems like the networking for the remote builder isn’t working all that well for you. You will have better success if you install and run Docker locally, then deploy with fly deploy --remote-only.

I don’t know where you’re connecting from, but remote builders work best with reliable and fast internet. They’re kind of fragile when internet is unstable. So if you’re a long distance away from one of our gateways, you’re going to end up with a worse experience.

@enaia this error is probably caused by an auth error.

Try running fly agent stop, and then fly auth login. Then see if that helps!

2 Likes

Hmm this is a deploy using your GitHub action and an API token.

It looks like you’re trying to push to the wrong app repository: registry.fly.io/production

Are you running flyctl deploy or building+pushing separately?

Hmm odd - we haven’t changed our CI config at all, and this worked until last Friday. This is what we’re doing. We added the --local-only after the deploy started failing, based on an earlier suggestion in this thread.

  deploy:
    runs-on: ubuntu-latest
    needs: [int_test, build]
    if: github.ref == 'refs/heads/main'
    env:
      FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}
    steps:
      - name: Checkout
        uses: actions/checkout@v2
        with:
          ref: main # Check out main instead of the latest commit
          fetch-depth: 0 # Checkout the whole branch

      - uses: superfly/flyctl-actions@1.1
        with:
          args: "deploy --local-only -c fly.production.toml"

I just realized my mistake, I found your app so that isn’t the problem. We’ll keep looking and see what might be happening.

Have you revoked a personal access token recently? Can you try creating a new one and updating the secret in GitHub?

OK just verifying that you were referring to the FLY_API_TOKEN, not a github personal access token? Anyway, I hadn’t revoked either but I just generated a new fly token via flyctl auth token and updated the FLY_API_TOKEN secret, and deploys are working again.