"no space left on device" when remote building

I’ve been using a remote builder successfully for many builds all day. Suddenly, my build starts failing with:

Error error building: error rendering build status stream: open /data/docker/...: no space left on device

Do you have some kind of space leak maybe?

We have logic in place that should clear space in the event that too much has been allocated. It seems like it was unable to reclaim space in this case? Hmm.

Every organization has its own builder. They come with a 50GB volume.

The easiest way to fix the issue is to destroy your builder. Find the name of your builder for the organization that owns the app you’re building by looking for an app that starts with fly-builder- and destroy it with flyctl apps destroy fly-builder-xyz. The next build will create a new builder with a fresh volume.

The harder way to is start a build, let it fail and flyctl ssh console -a fly-builder-xyz, and docker system prune --all --volumes --force to delete everything. To be honest, it’s probably not worth it to do that. I’d be curious to see what’s taking up so much space though.

Builds have been failing consistently or just the one?

1 Like

I tried to build several times and it failed with out of space errors at different point in the build every time. I deleted the builder and now it’s working.

1 Like