With a CI system which makes it trickier to debug.
I got the idea about destroying the builder app to ensure it got a new, latest one from here, as I recall, from one of my prior run-ins with it:
So that’s been my go-to “fix” as it randomly does work afterwards. But it’s not ideal and that may be a coincidence.
You may be on to something with the list command though as first, it seemed the builder app has no machines …
~ $ fly machines list -a fly-builder-solitary-shape-9303
ID IMAGE CREATED STATE REGION NAME IP ADDRESS
~ $ fly apps list
NAME OWNER STATUS LATEST DEPLOY
...
fly-builder-solitary-shape-9303 name pending
...
Which would explain why it’s waiting forever for it. if it’s still pending. Interesting. It still is.
So I tried running that command again, and this time it does have a machine listed … but it still shows as pending …
~ $ fly apps list
NAME OWNER STATUS LATEST DEPLOY
...
fly-builder-solitary-shape-9303 name pending
...
~ $ fly machines list -a fly-builder-solitary-shape-9303
ID IMAGE CREATED STATE REGION NAME IP ADDRESS
89006c93 flyio/rchab 2022-01-17 15:26:37 +0000 UTC started iad wispy-rain-5546 fdaa:0:5d9:a7b:21e0:0:78bc:2
The latest deploy I just tried failed again, same issue. So maybe it is the ‘pending’ that is the cause.
Or … maybe could it be the region? The builder is in iad, but the app I want to deploy has one instance … in lhr.
As for the CLI, it is the latest version, as I install the CLI fresh each time:
Executing command: /root/.fly/bin/fly version
WARN no config file found at /root/.fly/config.yml
flyctl v0.0.282 linux/amd64 Commit: 02c46ec BuildDate: 2022-01-12T17:44:39Z
I also tried running those suggested commands from the CI script too:
Executing command: /root/.fly/bin/fly agent stop
WARN no config file found at /root/.fly/config.yml
Error can't connect to agent: dial unix /root/.fly/fly-agent.sock: connect: no such file or directory
… no agent, but I guess perhaps that is correct there isn’t.
Executing command: /root/.fly/bin/fly dig txt _apps.internal
Reading environment variable exporting file contents.
Reading environment variable exporting file contents.
WARN no config file found at /root/.fly/config.yml
Error get app: Could not resolve App
Didn’t like that either. Failed with an error.
The same process (remote, deploy, using builder) did work two days ago when I last did a deploy, using the same CI process, server etc, so that’s why it’s strange. If it was a network issue you’d think it would never work or always work. Hmm.
Maybe it is the builder stuck at pending and/or the region it is in. Maybe deleting ones have happened to either move one into running and/or make one in lhr. I’ve not looked at the builder app status or region before.