I have an app I’m in the process of writing, which will require a lot of disk space. I don’t yet know how this will scale, so what I’m wondering is how quickly you can, through the API, create a new volume, attach it to a machine, and then remove it afterwards?
The app would run in the background, dispatching jobs, so it’s okay if it takes a few seconds. However, I’m curious if anyone else uses on-the-fly volume attaching/detaching, and what your experience with it has been?
Hi… Making a volume larger is easy, but the other direction is not very smooth or convenient, unfortunately…
https://fly.io/docs/volumes/volume-manage/#extend-a-volume
https://fly.io/docs/machines/api/volumes-resource/#extend-a-volume
https://fly.io/docs/reference/configuration/#auto_extend_size_increment
Generally, you can’t remove the volume from a running Machine. You’d need to stop and then destroy it [the Machine] first. That’s typically ok, since Machines are designed to be disposable, and you’ll almost always want ≥2 of them, given that that’s more or less an architectural norm on the Fly.io platform. (Thus, the second Machine can handle traffic while the first is taken offline for overhaul.)
Moreover, if you wanted to preserve some of the now-too-large volume’s data, then you would need to perform that copying, to a newly created smaller volume, yourself. This can be handled via rsync or the like, but is another inconvenience.
Having said that, I attach new Machines to older volumes pretty frequently in experiments, and have never seen surprises.
Hope this helps!
Thanks, that’s very helpful. The data would be ephemeral. I need to download from S3, extract a zip file, do some file conversion and other stuff, upload back to S3, and then delete all the local data.
I suppose then what I could do is have like a library of disks attached, and store the state in the database of which disk is in use, and mark it as available when it’s ready for use.
Anyway, thanks for the info!
Could you use Tigris + FUSE instead of volumes?
Hmm… if you’re using AWS S3, I wonder if mapping S3 to your filing system via FUSE would be preferable.
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.