Distributed Volumes in Fly

[halfer]

Just spotted a library on another thread; another S3/fs-mount option:

Best practice for large machine <> Tigris transfer?

I’ve got a project where I’d like to process some large files that are stored on Tigris. (Large = 1-10gb). My process is: on HTTP trigger, copy file from Tigris to fly machine (currently using Node AWS S3 SDK v3) do some processing to make more derivative files (primarily at the commandline with tiffsplit and vips) send all derivative files back to Tigris delete everything locally. Or, at least, that’s the plan. However, download performance from Tigris feels painfully slow, and that’s befo…