My second day trying out Fly, and my second day with deployments randomly failing because the remote builder has some problem with volume creation (see below).
This gives a very bad impression. Like: “Maybe I should just look at something else” bad. Whereas in general I’m really impressed with Fly.
Is this common? (I see this, which could be another example.)
Is there anything one can do to fix it other than wait? (Other than running docker locally, which I think is an option…)
We’re investigating issues related to volume creation on this host. Volume creation and deploys are temporarily disabled on this host, but already running apps will continue to work normally.
Service Interruption
34 minutes ago
We’re investigating issues related to volume creation on this host. Volume creation and deploys are temporarily disabled on this host, but running apps will continue to work.
You are indeed correct: building locally is an option. If you have Docker running, try fly deploy --local-only. As the flag suggests, that will build an image of your app locally and then push it up to Fly. That can be slower, but it depends on your connection.
However that particular error message suggests it may not actually be the builder that’s the issue here. Fly’s VMs run on hosts (servers). The error suggests that host can’t add new volumes (for example if it reaches capacity, but it’s probably some other issue). I don’t know why. As a temporary bypass you could deploy to a different region . That would ensure your app won’t be on that particular host/server. If it is the builder trying to create the new volume, yep, the local build would be the fix/bypass for that.
Thanks. The problem was definitely the builder, not the app machine; those messages (the screenshot) were for the builder machine. The app machines were fine.
Fly finally resolved the issue after ~6 hours – which is to say, overnight, as I had to give up on it at the end of the work day (got resolved at about 11 p.m.). Same thing the first time, I had to give up on it at around 5:00 p.m. (from memory) and it got resolved about three hours later.
I wonder if the experience is better on > Hobby plans. At least I’d be able to email support.
If it happens again you could try and destroy the builder. After all, it is just another app. Then when you next deploy, Fly-bot would see you don’t have a builder and so make you a new one. Which (you’d assume) would be created on a host that wasn’t having issues .
. That would ensure your app won’t be on that particular host/server. If it is the builder trying to create the new volume, yep, the local build would be the fix/bypass for that.
)