Automatically delete volume when machine stops

Questions / Help
,
1

Hi, I’m currently evaluating fly.io for a potential use-case.

Is it possible to automatically delete a volume, when the associated machine stops? I’m trying to utilize Fly.io Machines for user tasks that run once. This requires starting machines for a task and then deleting them once the task is done. It seems that is already possible: Fly machines automatically delete? - #2 by JP_Phillips

My question is if this is possible with volumes as well? Since ephemeral storage doesn’t seem to be extensible on Fly.io Machines (Scaling Ephemeral Disk - #2 by kurt), I would like to go with an automatically created volume for each machine, which would have to be automatically removed at the end of the machine’s lifecycle. I wasn’t able to find anything on this topic so far.

2

Hi!

This is not directly possible; by definition a volume is to be used when you want to persist information across machine starts, since as you observed, the machine’s own rootfs is ephemeral, so auto-deleting a volume when the machine dies is not a supported native operation.

You can probably hack something equivalent by using truly ephemeral machines (fly machine run --rm --volume volume_id:path) and having your image run something at start-up (some kind of wrapper script) that wipes the volume before starting the task in question.

  • Daniel
3

So for my use case I would have to:

  • Create a volume
  • Create a machine
  • Attach the volume to the machine
  • Run my task and wait for the machine to shut down
  • Destroy the machine
  • Destroy the volume

I’m still very new to the platform, so I’m not familiar with all the primitives, but that seems like a lot of work just to extend the space I can do file operations on. I’m currently using EC2 instances on AWS and I had no problem extending the disk space there.

I understand that volumes are meant for persistent storage, but it seems like almost an oversight that there is no way to extend ephemeral storage. Is there any other way to make this happen more easily?

My main concern is unnecessary cost associated with potential useless volumes in my account. I would like to have true scale-to-zero for this application.

4

yikes, when you put it like that…

$ VOL_ID=fly volume create foo -a $APP_NAME --region yyz --size 1 --json --yes | jq -r .id
$ fly machine run -a $APP_NAME --detach --rm --region yyz --volume $VOL_ID:/data ubuntu:24.04 -- ls -la /data

and then cron this, or run it after each machine creation to reap old unattached volumes:

$ fly volumes list -a $APP_NAME --json | jq -r '.[] | select(.attached_machine_id==null) | .id' | xargs fly volume delete --yes -a $APP_NAME

Let me know if this helps with volume management. Other than that, there’s no way to “extend” rootfs on Fly.

Out of curiosity, how much space do you need for your machine tasks?

  • Daniel
5

Thank you very much for the example, I’ll see if that works for my use-case.

I was targeting about 20 GB for these tasks.

6

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.