Today's bugs

$ fly version update
Update available 0.0.189 → 0.0.229
Update with flyctl version update

/bin/bash -c
Running automatic update [curl -L “https://fly.io/install.sh” | sh -s prerel]
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1411 0 1411 0 0 8109 0 --:–:-- --:–:-- --:–:-- 8109
Error: Unable to find a flyctl release for Linux/x86_64/prerel - see github.com/superfly/flyctl/releases for all versions
exit status 1

Updated by reinstalling.

Now I can’t deploy

$ fly deploy
Deploying muddy-mountain-2875
==> Validating app configuration
--> Validating app configuration done

WARN Remote builder did not start on time. Check remote builder logs with `flyctl logs -a fly-builder-solitary-meadow-8212`
Error error connecting to docker: remote builder app unavailable

Giving up for the day disappointed. I got 3 errors in a row while following the official instructions

Broken link:

Smokescreen - Sanitize outbound web requests · Fly links to https://github.com/fly-apps/smokescreen-example/, which is a 404

It should be GitHub - fly-apps/smokescreen: An example of deploying Smokescreen on Fly.io

We’re working on the “can’t connect to remote builder” issue, there are times when flyctl can’t seem to connect to wireguard. You can follow these issues:

That install error is interesting, it was trying to install a pre-release version, but we don’t have a current pre-release. Please let us know if the update fails from the current stable version next time.

I’ve still not been able to deploy my app. I’ve tried a few times and it times out every time.

Will you try running: LOG_LEVEL=debug fly deploy and post the output here?


You can work around this with a local Docker install. We’ll use that to build if it’s running, and the remainder of the deploy should work fine. There seems to be a problem getting you connected to a builder.

I wonder if these are the same issue.

Likely, we are having the hardest time replicating but it’s almost definitely an issue with connecting via Wireguard.

1 Like
DEBUG Loaded flyctl config from/home/gavin/.fly/config.yml
DEBUG Loading app config from/home/gavin/smokescreen/fly.toml
DEBUG Working Directory: /home/gavin/smokescreen
DEBUG App Config File: /home/gavin/smokescreen/fly.toml
Deploying muddy-mountain-2875
==> Validating app configuration
DEBUG --> POST https://api.fly.io/graphql {{"query":"query($appName: String!, $definition: JSON!) { app(name: $appName) { parseConfig(definition: $definition) { definition valid errors services { description } } } }","variables":{"appName":"muddy-mountain-2875","definition":{}}}
}
DEBUG <-- 200 https://api.fly.io/graphql (240.67ms) {"data":{"app":{"parseConfig":{"definition":{"kill_timeout":5,"kill_signal":"SIGINT","experimental":{"allowed_public_ports":[]},"services":[],"env":{}},"valid":true,"errors":[],"services":[]}}}}
--> Validating app configuration done
DEBUG trying local docker daemon
DEBUG Local docker daemon unavailable
DEBUG trying remote docker daemon
DEBUG Trying 'Buildpacks' strategy
DEBUG no buildpack builder configured, skipping
DEBUG result image:<nil> error:<nil>
DEBUG Trying 'Dockerfile' strategy
DEBUG --> POST https://api.fly.io/graphql {{"query":"mutation($input: EnsureRemoteBuilderInput!) { ensureRemoteBuilder(input: $input) { url, app { name } } }","variables":{"input":{"appName":"muddy-mountain-2875","organizationId":null}}}
}
DEBUG <-- 200 https://api.fly.io/graphql (519.77ms) {"data":{"ensureRemoteBuilder":{"url":"https://fly-builder-solitary-meadow-8212.fly.dev","app":{"name":"fly-builder-solitary-meadow-8212"}}}}
DEBUG Remote Docker builder host: tcp://fly-builder-solitary-meadow-8212.internal:2375
Waiting for remote builder fly-builder-solitary-meadow-8212... starting ⣾ DEBUG --> POST https://api.fly.io/graphql {{"query":"query ($appName: String!) { app(name: $appName) { id name hostname deployed status version appUrl organization { id slug } services { description protocol internalPort ports { port handlers } } ipAddresses { nodes { id address type createdAt } } } }","variables":{"appName":"muddy-mountain-2875"}}
}
DEBUG --> POST https://api.fly.io/graphql {{"query":"query($appName: String!, $showCompleted: Boolean!) { appstatus:app(name: $appName) { id name deployed status hostname version appUrl organization { slug } deploymentStatus { id status version description placedCount promoted desiredCount healthyCount unhealthyCount } allocations(showCompleted: $showCompleted) { id idShort version latestVersion status desiredStatus totalCheckCount passingCheckCount warningCheckCount criticalCheckCount createdAt updatedAt canary region restarts healthy privateIP checks { status output name } } } }","variables":{"appName":"fly-builder-solitary-meadow-8212","showCompleted":false}}
}
Waiting for remote builder fly-builder-solitary-meadow-8212... starting ⣽ DEBUG <-- 200 https://api.fly.io/graphql (320.13ms) {"data":{"appstatus":{"id":"fly-builder-solitary-meadow-8212","name":"fly-builder-solitary-meadow-8212","deployed":true,"status":"dead","hostname":"fly-builder-solitary-meadow-8212.fly.dev","version":0,"appUrl":"https://213.188.211.112","organization":{"slug":"personal"},"deploymentStatus":null,"allocations":[]}}}
DEBUG <-- 200 https://api.fly.io/graphql (392.41ms) {"data":{"app":{"id":"muddy-mountain-2875","name":"muddy-mountain-2875","hostname":"muddy-mountain-2875.fly.dev","deployed":false,"status":"pending","version":0,"appUrl":null,"organization":{"id":"jw9aBN7LJX9QMtmnmORXKjDRX9u9PD","slug":"personal"},"services":[{"description":"TCP 80/443 ⇢ 8080","protocol":"TCP","internalPort":8080,"ports":[{"port":80,"handlers":["HTTP"]},{"port":443,"handlers":["TLS","HTTP"]}]}],"ipAddresses":{"nodes":[]}}}}
DEBUG creating wireguard config for org personal
DEBUG Found WireGuard state in local configuration
DEBUG Establishing WireGuard connection (interactive-thompson-gavinwahl-gmail-com-893)
Waiting for remote builder fly-builder-solitary-meadow-8212... starting ⣟ DEBUG --> POST https://api.fly.io/graphql {{"query":"query($appName: String!, $showCompleted: Boolean!) { appstatus:app(name: $appName) { id name deployed status hostname version appUrl organization { slug } deploymentStatus { id status version description placedCount promoted desiredCount healthyCount unhealthyCount } allocations(showCompleted: $showCompleted) { id idShort version latestVersion status desiredStatus totalCheckCount passingCheckCount warningCheckCount criticalCheckCount createdAt updatedAt canary region restarts healthy privateIP checks { status output name } } } }","variables":{"appName":"fly-builder-solitary-meadow-8212","showCompleted":false}}
}
Waiting for remote builder fly-builder-solitary-meadow-8212... starting ⣯ DEBUG <-- 200 https://api.fly.io/graphql (218.38ms) {"data":{"appstatus":{"id":"fly-builder-solitary-meadow-8212","name":"fly-builder-solitary-meadow-8212","deployed":true,"status":"running","hostname":"fly-builder-solitary-meadow-8212.fly.dev","version":0,"appUrl":"https://213.188.211.112","organization":{"slug":"personal"},"deploymentStatus":null,"allocations":[]}}}
Waiting for remote builder fly-builder-solitary-meadow-8212... starting ⣻ DEBUG --> POST https://api.fly.io/graphql {{"query":"query($appName: String!, $showCompleted: Boolean!) { appstatus:app(name: $appName) { id name deployed status hostname version appUrl organization { slug } deploymentStatus { id status version description placedCount promoted desiredCount healthyCount unhealthyCount } allocations(showCompleted: $showCompleted) { id idShort version latestVersion status desiredStatus totalCheckCount passingCheckCount warningCheckCount criticalCheckCount createdAt updatedAt canary region restarts healthy privateIP checks { status output name } } } }","variables":{"appName":"fly-builder-solitary-meadow-8212","showCompleted":false}}
}
DEBUG <-- 200 https://api.fly.io/graphql (195.87ms) {"data":{"appstatus":{"id":"fly-builder-solitary-meadow-8212","name":"fly-builder-solitary-meadow-8212","deployed":true,"status":"running","hostname":"fly-builder-solitary-meadow-8212.fly.dev","version":0,"appUrl":"https://213.188.211.112","organization":{"slug":"personal"},"deploymentStatus":null,"allocations":[]}}}
Waiting for remote builder fly-builder-solitary-meadow-8212... starting ⣯ DEBUG --> POST https://api.fly.io/graphql {{"query":"query($appName: String!, $showCompleted: Boolean!) { appstatus:app(name: $appName) { id name deployed status hostname version appUrl organization { slug } deploymentStatus { id status version description placedCount promoted desiredCount healthyCount unhealthyCount } allocations(showCompleted: $showCompleted) { id idShort version latestVersion status desiredStatus totalCheckCount passingCheckCount warningCheckCount criticalCheckCount createdAt updatedAt canary region restarts healthy privateIP checks { status output name } } } }","variables":{"appName":"fly-builder-solitary-meadow-8212","showCompleted":false}}
}
Waiting for remote builder fly-builder-solitary-meadow-8212... starting ⣷ DEBUG <-- 200 https://api.fly.io/graphql (155.07ms) {"data":{"appstatus":{"id":"fly-builder-solitary-meadow-8212","name":"fly-builder-solitary-meadow-8212","deployed":true,"status":"running","hostname":"fly-builder-solitary-meadow-8212.fly.dev","version":0,"appUrl":"https://213.188.211.112","organization":{"slug":"personal"},"deploymentStatus":null,"allocations":[]}}}
Waiting for remote builder fly-builder-solitary-meadow-8212... starting ⢿ DEBUG --> POST https://api.fly.io/graphql {{"query":"query($appName: String!, $showCompleted: Boolean!) { appstatus:app(name: $appName) { id name deployed status hostname version appUrl organization { slug } deploymentStatus { id status version description placedCount promoted desiredCount healthyCount unhealthyCount } allocations(showCompleted: $showCompleted) { id idShort version latestVersion status desiredStatus totalCheckCount passingCheckCount warningCheckCount criticalCheckCount createdAt updatedAt canary region restarts healthy privateIP checks { status output name } } } }","variables":{"appName":"fly-builder-solitary-meadow-8212","showCompleted":false}}
}
Waiting for remote builder fly-builder-solitary-meadow-8212... starting ⣟ DEBUG <-- 200 https://api.fly.io/graphql (399.21ms) {"data":{"appstatus":{"id":"fly-builder-solitary-meadow-8212","name":"fly-builder-solitary-meadow-8212","deployed":true,"status":"running","hostname":"fly-builder-solitary-meadow-8212.fly.dev","version":0,"appUrl":"https://213.188.211.112","organization":{"slug":"personal"},"deploymentStatus":null,"allocations":[{"id":"076fa4f7-b151-8a54-78ad-258a0be1d7d8","idShort":"076fa4f7","version":0,"latestVersion":true,"status":"pending","desiredStatus":"run","totalCheckCount":0,"passingCheckCount":0,"warningCheckCount":0,"criticalCheckCount":0,"createdAt":"2021-08-05T19:56:08Z","updatedAt":"2021-08-05T19:56:08Z","canary":false,"region":"iad","restarts":0,"healthy":true,"privateIP":null,"checks":[]}]}}}
Waiting for remote builder fly-builder-solitary-meadow-8212... connecting ⣻ DEBUG Remote builder unavailable, retrying in 200ms (err: Cannot connect to the Docker daemon at tcp://fly-builder-solitary-meadow-8212.internal:2375. Is the docker daemon running?)
Waiting for remote builder fly-builder-solitary-meadow-8212... connecting ⣻ DEBUG Remote builder unavailable, retrying in 232.492502ms (err: Cannot connect to the Docker daemon at tcp://fly-builder-solitary-meadow-8212.internal:2375. Is the docker daemon running?)
Waiting for remote builder fly-builder-solitary-meadow-8212... connecting ⢿ DEBUG Remote builder unavailable, retrying in 234.471878ms (err: Cannot connect to the Docker daemon at tcp://fly-builder-solitary-meadow-8212.internal:2375. Is the docker daemon running?)
Waiting for remote builder fly-builder-solitary-meadow-8212... connecting ⡿ DEBUG Remote builder unavailable, retrying in 301.47562ms (err: Cannot connect to the Docker daemon at tcp://fly-builder-solitary-meadow-8212.internal:2375. Is the docker daemon running?)
Waiting for remote builder fly-builder-solitary-meadow-8212... connecting ⣟ DEBUG Remote builder unavailable, retrying in 205.567777ms (err: Cannot connect to the Docker daemon at tcp://fly-builder-solitary-meadow-8212.internal:2375. Is the docker daemon running?)
Waiting for remote builder fly-builder-solitary-meadow-8212... connecting ⣯ DEBUG Remote builder unavailable, retrying in 421.349917ms (err: Cannot connect to the Docker daemon at tcp://fly-builder-solitary-meadow-8212.internal:2375. Is the docker daemon running?)
Waiting for remote builder fly-builder-solitary-meadow-8212... connecting ⣷ DEBUG Remote builder unavailable, retrying in 446.048663ms (err: Cannot connect to the Docker daemon at tcp://fly-builder-solitary-meadow-8212.internal:2375. Is the docker daemon running?)
Waiting for remote builder fly-builder-solitary-meadow-8212... connecting ⣽ DEBUG Remote builder unavailable, retrying in 648.497325ms (err: Cannot connect to the Docker daemon at tcp://fly-builder-solitary-meadow-8212.internal:2375. Is the docker daemon running?)
Waiting for remote builder fly-builder-solitary-meadow-8212... connecting ⣟

Do you wanna give this a try?

Ok, I tried this. There was a reference in the ‘wire_guard_state’ section of fly config that referenced a name that didn’t exist.

I then deleted .fly/config and logged in.

I then try to deploy and got a failure:
Error error connecting to docker: error creating wireguard config: An unknown error occured.

I then used fly wireguard list and fly wireguard remove to remove all existing gateways/peers/wireguard/whatever you call them. I don’t understand why you have to manage these yourself.

Then a deploy worked

1 Like