I’ve been running an automated Fly.io deployment setup with GitHub workflows, and it’s been working as expected. This is how a production deployment log usually looks like, running
flyctl deploy --now --remote-only to deploy a new app version for every new commit on the production branch (and some other stuff due to this other bug, but that’s not relevant). Since the day before yesterday, this has changed, and I’m encountering a variety of errors before/after connecting to the remote builder:
- “no space left on device” log
- “remote builder app unavailable” log
- “context deadline exceeded” (this happens when trying to deploy from my local machine to the remote builder)
I went through a couple of other reports on these error messages and ran
$ fly ping -o lsglab
Which didn’t output anything (unlike my personal builder). After running
fly wireguard list, (I think) I found the culprit: my personal builder only has one
interactive- connection to my own computer, but the organization one that is causing problems has 91. I have tried removing a couple of them individually, one-by-one using
fly wireguard remove, and now I’m getting a “remote builder app unavailable”/“builder did not start in time” error too.
How can I get this builder to work again? Is there a command to clear all WireGuard peers? (Would that help?) Should I and How can I stop persisting all these automated GitHub runner environments as WireGuard peers?