Remote builder connection failing

Since yesterday, I’ve been unable to deploy with a remote builder (using the Paketo Go buildpack). It worked at first, in the morning yesterday (European time), but I now get:

Deploying lingering-sun-7724
==> Validating app configuration
--> Validating app configuration done

WARN Remote builder did not start on time. Check remote builder logs with `flyctl logs -a fly-builder-quiet-sun-4345`
Error remote builder app unavailable

The builder instance does start (it’s the “connecting” that fails/times out after a while), and checking the logs just gives:

2021-07-14T13:03:27.309939037Z app[d8239413] iad [info] time="2021-07-14T13:03:27.308897160Z" level=debug msg="checking docker activity"
2021-07-14T13:03:27.311490743Z app[d8239413] iad [info] time="2021-07-14T13:03:27.310037820Z" level=debug msg="Calling GET /v1.41/containers/json?filters=%7B%22status%22%3A%7B%22running%22%3Atrue%7D%7D&limit=0"
2021-07-14T13:03:28.312314296Z app[d8239413] iad [info] time="2021-07-14T13:03:28.311062489Z" level=debug msg="checking docker activity"
2021-07-14T13:03:28.314506741Z app[d8239413] iad [info] time="2021-07-14T13:03:28.312342380Z" level=debug msg="Calling GET /v1.41/containers/json?filters=%7B%22status%22%3A%7B%22running%22%3Atrue%7D%7D&limit=0"
(etcetera, repeating every second…)

Destroying the builder app doesn’t help – the same thing happens with the new one.


Are you still having this issue? What OS are you using?

Yes, still the same issue just now. I’m on macOS 11.4, with up-to-date flyctl.

Will you try running flyctl agent restart? That’s a relatively recent command, so if it errors you’re on an older version.

If you are on an older version, is there any chance you’re also connected to a VM with fly ssh console? Before yesterday, you couldn’t connect to two places at once, so an open SSH connection would make a builder connection fail.

The flyctl agent restart command appears to have succeeded. No, I’m not currently connected to any VM using ssh. I could try from a Linux machine…

I think that’s an agent bug, then, if that command works you’re already using the build that supports multiple connections. We’ll see if we can figure out what’s up with it.

It worked from a Linux machine.