[LiteFS] How to work with multiple databases?

1

I am using Pocketbase.io, and I know I can only allow the replicas to be read only.

I have non writing apps that will share the Consul Lease, and point to the same LiteFS Cloud instance.

Pocketbase creates both a data.db and logs.db database.

When using a static lease locally, I never specify data.db, but with the consul lease option, I have to specify the data.db in the proxy configuration.

proxy:
  addr: ":8080"
  target: ":8055"
  db: "data.db"
  passthrough:
    - "*.ico"
    - "*.png"

My question is, is there a way to specify multiple db files in the proxy?

Just guessing, but I assume my options are -

Multiple in One Proxy

proxy:
  addr: ":8080"
  target: ":8055"
  db: 
    - "data.db"
    - "logs.db"
  passthrough:
    - "*.ico"
    - "*.png"

Two Proxy Defines

proxy:
  addr: ":8080"
  target: ":8055"
  db: "data.db"
  passthrough:
    - "*.ico"
    - "*.png"

proxy:
  addr: ":8080"
  target: ":8055"
  db: "logs.db"
  passthrough:
    - "*.ico"
    - "*.png"

The first one makes the most sense to me, has anyone solve this? I know for static leasing it just works, so it’s a little strange the proxy isn’t just setting the data directory, and instead the specific database.

2

The proxy uses the referenced database to ensure consistency between the time a user makes a write to the primary and then a follow up request to a replica. I assume the data.db is the main data storage database but what is the logs.db for?

If logs.db is just used for something behind-the-scenes like request logging then you probably don’t need to track it for consistency for the end users and you can just specify data.db in the proxy.

3

I had this same issue when trying to setup NocoDB, which does have a meta.db that should be separate and is needed across environments.

My concern is that logs.db will try to be written too in a read-only mode and cause issues.

I don’t think i have access to a setting to disable logging.

While I don’t know that I need to backup or synced, because it is a lower concern, it would be a issue if the act of logging causes issues on other environments because the POST request does not route correctly.

Is there a way to handle multiple DBs? Your answer didn’t make that totally clear.

4

There’s not a way with LiteFS to write to databases on replicas. I think Pocketbase will need to add a way to disable writes to the logs.db for GET requests if you want to use the proxy.

The proxy just handles one database. Your request can alter multiple databases but consistency is only tracked for one of them. It sounds like logs.db consistency isn’t that important so just setting the proxy.db to data.db should be fine.

Ultimately, I think the bigger issue is avoiding logs.db writes on replicas. If every GET request is going to log to that database then you’d need to redirect every request to the primary which defeats the purpose of LiteFS. :slight_smile:

1 Like
5

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.