Unable to import remote instance LiteFS

Hey, I have a remix application that has a running sqlite.db under a LiteFS cluster. Here is the docker file that gets it up and running. This is working fine

# This file is moved to the root directory before building the image

# base node image
FROM node:18-bookworm-slim as base

# set for base and all layer that inherit from it
ENV NODE_ENV production

# Install openssl for Prisma
RUN apt-get update && apt-get install -y fuse3 openssl sqlite3 ca-certificates

# Install all node_modules, including dev dependencies
FROM base as deps

WORKDIR /myapp

ADD package.json package-lock.json .npmrc ./
ADD ./other/patches ./other/patches
RUN npm install --include=dev

# Setup production node_modules
FROM base as production-deps

WORKDIR /myapp

COPY --from=deps /myapp/node_modules /myapp/node_modules
ADD package.json package-lock.json .npmrc ./
RUN npm prune --omit=dev

# Build the app
FROM base as build

WORKDIR /myapp

COPY --from=deps /myapp/node_modules /myapp/node_modules

ADD prisma .
RUN npx prisma generate

ADD . .
RUN npm run build

# Finally, build the production image with minimal footprint
FROM base

ENV FLY="true"
ENV LITEFS_DIR="/litefs/data"
ENV DATABASE_FILENAME="sqlite.db"
ENV DATABASE_PATH="$LITEFS_DIR/$DATABASE_FILENAME"
ENV DATABASE_URL="file:$DATABASE_PATH"
ENV CACHE_DATABASE_FILENAME="cache.db"
ENV CACHE_DATABASE_PATH="/$LITEFS_DIR/$CACHE_DATABASE_FILENAME"
ENV INTERNAL_PORT="8080"
ENV PORT="8081"
ENV NODE_ENV="production"

# add shortcut for connecting to database CLI
RUN echo "#!/bin/sh\nset -x\nsqlite3 \$DATABASE_URL" > /usr/local/bin/database-cli && chmod +x /usr/local/bin/database-cli

WORKDIR /myapp

COPY --from=production-deps /myapp/node_modules /myapp/node_modules
COPY --from=build /myapp/node_modules/.prisma /myapp/node_modules/.prisma

COPY --from=build /myapp/server-build /myapp/server-build
COPY --from=build /myapp/build /myapp/build
COPY --from=build /myapp/public /myapp/public
COPY --from=build /myapp/package.json /myapp/package.json
COPY --from=build /myapp/prisma /myapp/prisma
COPY --from=build /myapp/app/components/ui/icon.svg /myapp/app/components/ui/icon.svg
COPY --from=build /myapp/app/components/ui/icon.tsx /myapp/app/components/ui/icon.tsx

# prepare for litefs
COPY --from=flyio/litefs:0.5.1 /usr/local/bin/litefs /usr/local/bin/litefs
ADD other/litefs.yml /etc/litefs.yml
RUN mkdir -p /data ${LITEFS_DIR}

ADD . .

CMD ["litefs", "mount"]

But, here is the dockerfile for my webscraper app I am trying to connect to the same sqlite db within the remix application. This part aint working

FROM zenika/alpine-chrome:with-playwright

USER root

## ADDED IN STUFF!
# Install LiteFS dependencies for Alpine
RUN apk add ca-certificates fuse3 sqlite

# Copy LiteFS binary
COPY --from=flyio/litefs:0.5 /usr/local/bin/litefs /usr/local/bin/litefs

ENV NODE_ENV production
WORKDIR /src

COPY package*.json ./
RUN npm ci --production=false

COPY . .
RUN npm run build

EXPOSE 8080

RUN apk add --no-cache curl

# Install dumb-init
ENV DUMB_INIT_VERSION=1.2.5
RUN wget -O /usr/local/bin/dumb-init \
    https://github.com/Yelp/dumb-init/releases/download/v${DUMB_INIT_VERSION}/dumb-init_${DUMB_INIT_VERSION}_x86_64 \
    && chmod +x /usr/local/bin/dumb-init

# Latest releases available at https://github.com/aptible/supercronic/releases
ENV SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.2.1/supercronic-linux-amd64 \
    SUPERCRONIC=supercronic-linux-amd64 \
    SUPERCRONIC_SHA1SUM=d7f4c0886eb85249ad05ed592902fa6865bb9d70 

ENV INTERNAL_PORT="8080" 
ENV PORT="8081"
ENV DATABASE_FILENAME="data.db"
ENV ADVERTISE_URL_REMIX="FAKE_URL :)"

RUN curl -fsSLO "$SUPERCRONIC_URL" \
 && echo "${SUPERCRONIC_SHA1SUM}  ${SUPERCRONIC}" | sha1sum -c - \
 && chmod +x "$SUPERCRONIC" \
 && mv "$SUPERCRONIC" "/usr/local/bin/${SUPERCRONIC}" \
 && ln -s "/usr/local/bin/${SUPERCRONIC}" /usr/local/bin/supercronic

COPY ./crontab ./crontab

RUN litefs import -name data.db -url $ADVERTISE_URL_REMIX ~/data

ENTRYPOINT ["dumb-init", "--"]
CMD ["litefs", "mount", "--ipc=host", "npm", "run", "start"]

So, last thing here is the litefs.yml which is pretty much similar within the remix and webscraper app. The only difference is the advertise-url, console url, and console key are copied from the remix app.

# Documented example: https://github.com/superfly/litefs/blob/dec5a7353292068b830001bd2df4830e646f6a2f/cmd/litefs/etc/litefs.yml
fuse:
  # Required. This is the mount directory that applications will
  # use to access their SQLite databases.
  dir: '${LITEFS_DIR}'

data:
  # Path to internal data storage.
  dir: '/data/litefs'

proxy:
  # matches the internal_port in fly.toml
  addr: ':${INTERNAL_PORT}'
  target: 'localhost:${PORT}'
  db: '${DATABASE_FILENAME}'

# The lease section specifies how the cluster will be managed. We're using the
# "consul" lease type so that our application can dynamically change the primary.
#
# These environment variables will be available in your Fly.io application.
lease:
  type: 'consul'
  candidate: ${FLY_REGION == PRIMARY_REGION}
  promote: true
  advertise-url: 'http://${FLY_ALLOC_ID_REMIX}.vm.${FLY_APP_NAME_REMIX}.internal:20202'

  consul:
    url: '${FLY_CONSUL_URL_FROM_REMIX}'
    key: 'litefs/${FLY_APP_NAME_REMIX}'

exec:
  - cmd: node ./other/setup-swap.js

  - cmd: npx prisma migrate deploy
    if-candidate: true

  - cmd: npm start

So, main thing is that I am definately connecting to the remote instance incorrectly and I think I am initiating LiteFS within the webscraper app incorrectly. If anyone can help I would greatly appreciate it.

Along with that if anyone has best practices for sharing the advertise-url, console url, and console key from the remix app into the webscraper that would be cool. I might follow this article How to scp files to and from Fly | Simon Willison’s TILs but I don’t know might be overkill.

Hi… This recent post on cross-application LiteFS sharing might be relevant as background reading:

https://community.fly.io/t/litefs-multiple-databases/15595/18

At a minimum, it will provide a functioning example of how advertise-url meshes.


Also, I don’t think your RUN litefs import can be made to work out, in the end.

As a general rule, you can’t do database operations in the Dockerfile. The builder has access neither to the volume nor to the members of the cluster, after all.

If data.db is an existing, third SQLite file that you want inlaid in LITEFS_DIR for use by the webscraper, then manually SSH into the current, running primary (within the “This is working fine” machine fleet) and litefs import it there. Any node created subsequently will automatically get a full copy from the primary, as part and parcel of its litefs mount.

(You’ll be able to see this happening in fly logs on the existing replicas, as well.)

Hope this helps a little!