Remote builder connection failing

Questions / Help
1

Since yesterday, I’ve been unable to deploy with a remote builder (using the Paketo Go buildpack). It worked at first, in the morning yesterday (European time), but I now get:

Deploying lingering-sun-7724
==> Validating app configuration
--> Validating app configuration done

WARN Remote builder did not start on time. Check remote builder logs with `flyctl logs -a fly-builder-quiet-sun-4345`
Error remote builder app unavailable

The builder instance does start (it’s the “connecting” that fails/times out after a while), and checking the logs just gives:

2021-07-14T13:03:27.309939037Z app[d8239413] iad [info] time="2021-07-14T13:03:27.308897160Z" level=debug msg="checking docker activity"
2021-07-14T13:03:27.311490743Z app[d8239413] iad [info] time="2021-07-14T13:03:27.310037820Z" level=debug msg="Calling GET /v1.41/containers/json?filters=%7B%22status%22%3A%7B%22running%22%3Atrue%7D%7D&limit=0"
2021-07-14T13:03:28.312314296Z app[d8239413] iad [info] time="2021-07-14T13:03:28.311062489Z" level=debug msg="checking docker activity"
2021-07-14T13:03:28.314506741Z app[d8239413] iad [info] time="2021-07-14T13:03:28.312342380Z" level=debug msg="Calling GET /v1.41/containers/json?filters=%7B%22status%22%3A%7B%22running%22%3Atrue%7D%7D&limit=0"
(etcetera, repeating every second…)

Destroying the builder app doesn’t help – the same thing happens with the new one.

Cheers,
Daniel

2

Are you still having this issue? What OS are you using?

3

Yes, still the same issue just now. I’m on macOS 11.4, with up-to-date flyctl.

4

Will you try running flyctl agent restart? That’s a relatively recent command, so if it errors you’re on an older version.

If you are on an older version, is there any chance you’re also connected to a VM with fly ssh console? Before yesterday, you couldn’t connect to two places at once, so an open SSH connection would make a builder connection fail.

5

The flyctl agent restart command appears to have succeeded. No, I’m not currently connected to any fly.io VM using ssh. I could try from a Linux machine…

6

I think that’s an agent bug, then, if that command works you’re already using the build that supports multiple connections. We’ll see if we can figure out what’s up with it.

7

It worked from a Linux machine.