So I have been building a thing over the past few months. I was supposed to be taking a break but this quickly turned into “scratching an itch”. I wanted to see how far I could take a userspace block device inspired by lsvd.
Elide is that userspace block device - leaning heavily on inspiration from the original Log-Structured Virtual Disk (lsvd) paper, but also bringing in other ideas - using content identity hashing to decouple the physical storage layer from the logical layout. This is a fast (NVMe fast) local block device acting as a cache for durable log-structured data in object storage (Tigris).
Fly.io machines support ublk (I think only recently with a kernel update?) and Elide takes full advantage of this. No nbd client-server to deal with - just /dev/elide/<vol> devices used as “regular” block devices. You can put an ext4 filesystem on it. If you set one up correctly you can give it to a container as a rootfs.
To get up and running you need a Linux box with a recent kernel (for ublk) and some object storage. A Fly.io app + provisioned Tigris is perfect for this.
There is a basic walkthrough here - elide/docs/quickstart.md at main · soulware/elide · GitHub
This takes you through launching/deploying to Fly.io + Tigris, creating a volume, writing to it and finally moving the volume between machine instances.