Background writes with LifeFS

Hello! Thanks again for LifeFS, it’s fab!

Fly’s HTTP proxy can be used to make it so POST etc requests which write to SQLite go via the primary node, however any writes made by the application independent to a HTTP request are not so easily routed to the primary. For example, an application could have a background thread which periodically writes to the database.

Here in the docs it says that LiteFS will create a $LITE_FS_DIR/.primary file when the node is the primary. Is the way to approach this to check this for the existance of this file before writing, to ensure that it is the primary?

Within Fly is there a way to send a HTTP request to the node that is the primary? That could be handy for indirectly performing a write perhaps!

Thanks,
Louis

Hello! I’m glad you’re enjoying LiteFS!

Yes, you can use that to check who the current primary is. We recently added an event stream endpoint to LiteFS (#401) which can also give you live updates about primary change status if you want to maintain that state within your application. The stream just returns newline-delimited JSON events.

You can send HTTP requests to other Fly nodes in your organization using the .internal addresses. By default, the .primary reports the hostname of the machine that is currently primary and this is also its machine ID. You can connect to that machine using:

$MACHINEID.vm.$APPNAME.internal
1 Like

We have docs section for the event stream if you’re interested in that. Probably easier to read than the PR. :slight_smile:

1 Like

Wonderful! Super helpful as always. Thanks!

Hello again! In my tests LiteFS 0.5 never creates the /litefs/.primary file, even though it has logged that it is the primary and is successfully replicating the database to LiteFS Cloud.

Is there something I need to do to enable this?

Thanks

The .primary file only exists on replicas. We found that it was easy for users to get in a circular loop if they were primary and tried to redirect based on the .primary file contents to themselves. Maybe it’s worth us adding a .role file or something so the node can check if it is the primary or not.

Ah ok! My mental model of how LiteFS works is wrong. I thought a replica was any node that is not the primary, I’m not sure what those terms mean now.

How can a node determine if it has write access to the database if it’s possible to have write permissions but .primary not be present?

edit: Ah! I’ve re-read the documentation and I got it backwards! I thought the file was present only for the primary, not only for the replicas.

Sorry for the confusion, thank you for the help!

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.