Can't complete flyctl deploy process - context deadline exceeded

I seem to have broken my ability to deploy a new app. Whenever I tried, it goes from
Waiting for remote builder fly-builder-summer-dawn-8254… connecting ⣟
to
Error error connecting to docker: unable to connect WireGuard tunnel: context deadline exceeded

When I check the status it says pending for the app and when I list apps, pending for the fly-builder as well. I destroyed my existing fly-builder and tried again, but it hasn’t made a difference. I thought something in my environment was causing a timeout, but I just tried to deploy the hellostatic-builtin example and that failed in the same way.

I was able to deploy an app last week! Please let me know what I need to tear down in order to get things moving and shaking again.

Thank you,
-James

What version of flyctl are you running?

flyctl v0.0.240 linux/amd64 Commit: 0ed4d26 BuildDate: 2021-09-15T16:52:14Z

It prompted me to upgrade this morning.

I wonder if your WireGuard peer got into a bad state. Let’s try removing them and see if it helps.

  1. run flyctl wg list <org> to see a list of your peers
  2. run flyctl wg remove <org> <name> on the ones with a prefix of “interactive-”.

Once that’s done, try deploying again.

Removed the WireGuard peer, tried to deploy again, still getting the context deadline exceeded.

I destroyed the fly-builder instance, just in case. The new one also seems to stay in Pending and I’m not able to get any logs from it, which is the behavior I was seeing from the old one.

@CallJCW can you post or DM me your builder app name?

Updated to 0.0.241, still getting this problem. I tried flyctl 0.0.238, which is the last time it worked, but the waiting for remote builder still timed out. I’m trying to nail down if it is a problem in my environment and if so, where.

I’m going to try another machine from within my network and see if that makes a difference.

Thank you,
-James

I’m currently seeing this error from CircleCI on about 30% of my deploys, but it’s inconsistent. If I retry the build, it works but then another deploy will fail a few minutes later.

I’m running the latest version of flyctl:

flyctl v0.0.256 linux/amd64 Commit: f39ea62 BuildDate: 2021-11-21T19:59:38Z

The failures look like this:

#!/bin/bash -eo pipefail
"${HOME}/.fly/bin/flyctl" deploy --access-token "${FLY_ACCESS_TOKEN}"

WARN no config file found at /home/circleci/.fly/config.yml
Deploying whatgotdone
==> Validating app configuration
--> Validating app configuration done
Services
TCP 80/443 ⇢ 8080
Waiting for remote builder fly-builder-spring-voice-5703... connecting ⣻ Error error connecting to docker: unable to connect WireGuard tunnel: context deadline exceeded



Exited with code exit status 126

My fly.toml is here: whatgotdone/fly.toml at 73c148aab96f65f29462473253b23fd156fd5c5a · mtlynch/whatgotdone · GitHub