Volume sharing within one app across scaled machines

[nativestranger]

We have a use case where we need to share a single volume between multiple scaled machines within the same app. We launch with one app, add a volume, and need it to be accessible for all other machines in the app.

This feature should unlock extremely heavy usage for fly.io soon

URGENT

Volume sharing within 1 app across scaled machines

Hello – I am trying to share 1 volume that has some data that I need to use for ffmpeg web server I am creating. Every time I try to scale up it creates a new empty volume. Is there a way to share a persistance volume with every other instances?

[lillian]

thanks for your interest! unfortunately, this is still not possible, and there aren’t any plans to add this feature (it would be very technically complex). we recommend storing your data in a S3-compatible service; there’s a few software that let you mount a S3 bucket as a POSIX filesystem if you need.

[halfer]

Just to add to Lillian’s advice, it’s worth understanding what a volume is, and why this feature request would effectively be a redesign:

A Fly Volume is a slice of an NVMe drive on the same physical server as the Machine on which it’s mounted and it’s tied to that hardware. Fly Volumes are similar to the disk inside your laptop, accessible from a mount point in your file system. Using volumes tied to hardware has both advantages and disadvantages.

(Source).

[nativestranger]

nice idea - is this easily doable on fly.io today? are there any other details I should know if I want to move forward with the S3/POSIX approach?

[lillian]

I’m not personally familiar with these methods, just aware of the concept. ideally, you would architecture your app to use a S3 remote rather than a POSIX filesystem since they tend to have different access patterns; if you really need a “local” mount you can look into juicefs, tigrisfs, or similar tools that can be found with a web search. we use JuiceFS internally for a few things, maybe that could work for you.

what is your use case, by the way?

[kurt]

What are you sharing between machines? Are you trying to do something like install a bunch of npm packages and have them all available everywhere? And I’m guessing you’re using tools that expect a filesystem?

[nativestranger]

Without revealing too much, “trying to do something like install a bunch of npm packages and have them all available everywhere” should satisfy for our requirement evaluation purposes

and we are indeed using using tools that expect a filesystem