Load balancing with litefs (no distribution between nodes)

I have gone through the Litefs implementation but i’m having some issues, the setup seems to work the exact same way as just using a volume. Only one copy of sqlite file is being updated. I have two machines, one a primary, one a replica:

Machines
PROCESS ID              VERSION REGION  STATE   ROLE    CHECKS  LAST UPDATED         
app     4d896d7c01d198  52      ord     started primary         2023-11-03T21:12:44Z
app     e784e1edbe5378  52      sea     started replica         2023-11-03T21:13:28Z

Below our my config files:

# fly.toml app configuration file generated for bumper-shopify on 2023-10-20T12:06:58-07:00
#
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#

app = "bumper-shopify"
primary_region = "ord"

[build]

[http_service]
  internal_port = 8081
  force_https = true
  auto_stop_machines = false
  auto_start_machines = true
  min_machines_running = 0
  processes = ["app"]

[env]
PORT = "8081"
SHOPIFY_APP_URL = "XXXXX"
SHOPIFY_API_KEY = "XXXXX"
SCOPES = "write_products,write_metaobject_definitions,write_metaobjects"

[mounts]
  source="data"
  destination="/data"

DOCKER FILE

FROM node:18-alpine

EXPOSE 8081
ENV DATABASE_URL="file:///data/db.sqlite"
WORKDIR /app
COPY . .

RUN npm install
RUN npm run build

# packages 
RUN apk update && apk upgrade
# for alpine-based images
RUN apk add ca-certificates fuse3 sqlite
COPY --from=flyio/litefs:0.5 /usr/local/bin/litefs /usr/local/bin/litefs

ADD ./litefs.yml /tmp/litefs.yml
ADD /prisma/dev.sqlite /tmp/db.sqlite
RUN cp /tmp/litefs.yml /etc/litefs.yml

ENTRYPOINT litefs mount

CMD ["npm", "run", "start"]

litefs.yml

# note: there should be one template for this, which you can update as necessary.
# note: had to leave the getting started guide and go to the lease management section bc not using fly
# note: lease.hostname and lease.advertise-url are not clear
# note: we should have an example docker-compose setup with everything working both with consul
# and static leases
# This directory is where your application will access the database.
fuse:
  dir: "/litefs"

# This directory is where LiteFS will store internal data.
# You must place this directory on a persistent volume.
data:
  dir: "/data"

# The lease section defines how LiteFS creates a cluster and
# implements leader election. For dynamic clusters, use the
# "consul". This allows the primary to change automatically when
# the current primary goes down. For a simpler setup, use
# "static" which assigns a single node to be the primary and does
# not failover.
lease:
  # Required. Must be either "consul" or "static".
  type: "static"

  # Required. The URL for this node's LiteFS API.
  # Should match HTTP port.
  advertise-url: "http://${PRIMARY_REGION}.${FLY_APP_NAME}.internal:20202"

  # Specifies whether the node can become the primary. If using
  # "static" leasing, this should be set to true on the primary
  # and false on the replicas.
  candidate: ${FLY_REGION == "ord"}

exec: 
  - cmd: "npm run start"

# proxy:
#   addr: ":8080"
#   target: "0.0.0.0:8081"
#   db: "db.sqlite"

You actually want your application to be writing to /litefs/ instead…

Think of /data/ as being more like the raw material / implementation details that LiteFS morphs into a usable cluster for you.

Oh, thanks a ton. I was missing that piece. I thought we had to create the primary db file in the persistent directory but it looks like there’s a dbs folder that’s being used for that purpose.

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.