LiteFS GET request containing writes + LiteFS with multiple processes

[mayailurus]

@dhakan

I think I might be able to serve the freshly fetched content from any of the nodes (primary/read) if the data doesn’t exist, and in that case enqueue a background job which will download and write it, after which it will be served from wherever. This ought to be quicker than replaying the request to the primary like I’m currently doing. It also means I can start using the LiteFS proxy, since my GET now only reads.

• What’s your thought on this approach?

That sounds pretty good, , if the external source can be relied on to return the same thing.

@dhakan

Am I expected to have a new volume created with duplicate data for each read replica? Won’t that be costly?

Machines never share volumes, so you do need multiple. Many people find that they can run 3 LiteFS nodes while staying within the free allowance.

But if you have more than 1GB of data already, then it will cost more, it’s true.

LiteFS replicates the entire database, rather than individual rows, which is both a strength and a weakness.

---

Tangent: People have asked in the past whether they could just store the non-primary-candidate nodes’ copies on those Machines’ built-in ephemeral root partitions instead. The LiteFS docs say that you “must place this directory on a persistent volume”, and Fly generally wishes that the root partition could be made strictly read-only. Moreover, the exact size and the like are not set in stone. I wouldn’t recommend using it for anything serious, even if it maybe does work at present…

---

@dhakan

I expected the DB to be connected through the /var/lib/litefs path, but it seems to be litefs/db. Where is this decided?

Your Node application should be writing (only) to /litefs/, but LiteFS will be transparently turning those into actions on /var/lib/litefs/. This is decided in litefs.yml, in the fuse and data sections.

FUSE is the mechanism that allows LiteFS to intercept reads from and writes to the /litefs/ filesystem hierarchy.

@dhakan

The poller runs every second, and it looks like it could potentially be picking up tasks from the queue (Redis in my case via node BullMQ) after it has lost its primary status. This will have the tasks fail. I guess this will be retried when the new worker starts up? Sounds like an edge case but still.

It is, but you definitely do want retry logic. Jobs can fail even in the single-node case, to be sure.

(Also, the LiteFS event stream is made to be kept open and read line by line, rather than polled. This isn’t super-convenient in bash; I was thinking more of Ruby or Racket.)