Unable to deploy NextJS app with sqlite db

sandervspl · April 12, 2023, 9:53am

I’ve followed all the steps in the docs. I simply cannot get this combination to work. I have an existing NextJS app (v13.3) that I want to deploy with an SQLite db.

My first attempt I ran fly launch and it created the docker and toml files. It deployed fine and looking at the monitor tab, it started the NextJS server. But when I went to the url of the deployment, the website would not show. It would load and load until the browser just errored with a “not responding” screen.

I thought maybe it was because I needed to add secrets. So I added them with fly secret set and added the NEXT_PUBLIC_ variables to my toml and dockerfile. Still didn’t work.

I created a volume for my SQLite db. In the docs it said to add something to my toml file, which I did

[mounts]
  source = "primenv_db"
  destination = "/mnt/primenv_db"

This did not work. I tried to run fly deploy but I was met with errors that this configuration wasn’t working. Something about not being connected to my app or something (I can’t remember exactly). The error instructed me to remove these lines or use the “machines API”. I don’t know what this means. I looked around on the docs on how to connect my volume to my app but I cannot find it. From the docs it is also not clear to me what destination means. Destination to what???

I tried to run fly deploy again and now it was giving me errors about not being able to destroy the machine and something about a lease??? I don’t know what that means? It had my name and an expiration date that kept changing. I couldn’t find anything in the docs regarding this error.

Then I said F it, I’ll start over. I deleted everything and started again with fly launch. It asked me if I wanted to use the existing toml file and I said yes. Then it did its thing and after it was done publishing the image it errored:

--> Pushing image done
image: registry.fly.io/primenv:deployment-01GXTCZNDHNQ82YX2TTCPEZZ0D
image size: 778 MB
Process groups have changed. This will:
 * create 1 "app" machine

No machines in group 'app', launching one new machine
Oops, something went wrong! Could you try that again?

It’s also created two apps: primenv and something with a random name fly-builder-damp-voice-8827. I can’t find what the second app is but this happens with every deploy.

Wasn’t expecting this to be so hard to deploy. I really want to try out Fly.io but this is discouraging.

So in short this is what I want to do:

Deploy my NextJS app
Setup an SQLite DB on a volume
And I guess I need to connect this volume to my NextJS app somehow

I hope I can get this sorted with y’all.

Edit: Okay I deleted my apps and the dockerfile and the toml file and ran fly deploy again. It deployed succesfully now but when I go to https://primenv.fly.dev nothing happens. The monitor tab says the server is running.

tj1 · April 12, 2023, 10:45am

Seems to be working?

sandervspl · April 12, 2023, 10:58am

That’s super strange. I cannot get it to work in my normal browser window but it does work in an incognito window. In a normal window it is stuck on loading until it errors with “not responding”.

Edit: Restarting my browser fixed it Thanks for checking.

I still need some help with setting up SQLite though. I am using drizzle-orm and sqlite on my local machine. I see in the drizzle-orm docs that they mention fly.io Litefs. My problem is that the fly.io docs keep mentioning Rails on almost every page regarding sqlite and litefs. It’s very confusing.

For example: how do I target my sqlite db on my volume? I have created and linked my volume to my app.

[mounts]
  source = "primenv_db"
  destination = "/primenv_db"

When I SSH to my app the primenv_db folder is empty. So I guess I shouldn’t target that directly. Do I need to use the DATABASE_URL? I tried both directly targeting the sqlite .db file and the DATABASE_URL but neither work.

DATABASE_URL = sqlite3:///primeenv_db/primenv_db/production.sqlite3

import Database from 'better-sqlite3';
import { drizzle } from 'drizzle-orm/better-sqlite3';
import { migrate } from 'drizzle-orm/better-sqlite3/migrator';

const sqlite = new Database(process.env.DATABASE_URL);
// For local I use this
// const sqlite = new Database('prime_env.db');

export const db = drizzle(sqlite);

Error:

TypeError: Cannot open database because the directory does not exist

sandervspl · April 12, 2023, 12:34pm

I’ve read through the LiteFS docs a little bit and I guess that is what I need to use. I have many questions, though.

For example this litefs.yml file

# This directory is where your application will access the database.
fuse:
  dir: "/litefs"

# This directory is where LiteFS will store internal data.
# You must place this directory on a persistent volume.
data:
  dir: "/var/lib/litefs"

How does this correlate to the volume that I created? I named my volume primenv_db . Should I point to that instead of /litefs ? And then the lease config. I only need 1 node that has write access. Do I still need this config? Where do I put it? I already have a lease config with type: ‘static’.

lease:
  type: "static"
  candidate: false
  hostname: "${PRIMARY_HOSTNAME}"
  advertise-url: "http://${PRIMARY_HOSTNAME}:20202"

The ORM I’m using (drizzle-orm) wants me to target a .db file. How do I know what the file I need to target is?

After reading this section it seems that I’m on the right track, but it’s still not working. I still don’t know how to target my DB. Also, my server is in an infinite restart loop it seems.

2023-04-12T12:49:50.593 proxy[4d89622c642058] ams [error] failed to connect to machine: gave up after 15 attempts (in 22.061193438s)
2023-04-12T12:49:52.533 proxy[4d89622c642058] ams [info] Starting machine
2023-04-12T12:49:52.867 app[4d89622c642058] ams [info] Starting init (commit: e878f33)...
2023-04-12T12:49:52.888 app[4d89622c642058] ams [info] Mounting /dev/vdb at /primenv_db w/ uid: 1001, gid: 65533 and chmod 0755
2023-04-12T12:49:52.892 app[4d89622c642058] ams [info] Preparing to run: `/bin/sh -c litefs mount -- primenv -dsn /primenv_db/db npm run start` as nextjs
2023-04-12T12:49:52.909 app[4d89622c642058] ams [info] 2023/04/12 12:49:52 listening on [fdaa:1:e59d:a7b:c6ef:5967:3bdf:2]:22 (DNS: [fdaa::3]:53)
2023-04-12T12:49:52.964 proxy[4d89622c642058] ams [info] machine started in 430.980682ms
2023-04-12T12:49:53.904 app[4d89622c642058] ams [info] Starting clean up.
2023-04-2T12:49:53.904 app[4d89622c642058] ams [info] Umounting /dev/vdb from /primenv_db
2023-04-12T12:49:54.906 app[4d89622c642058] ams [info] [ 2.133787] reboot: Restarting system
2023-04-12T12:49:55.885 runner[4d89622c642058] ams [info] machine has reached its max restart count (10)
2023-04-12T12:49:58.197 proxy[4d89622c642058] ams [info] waiting for machine to be reachable on 0.0.0.0:8080 (waited 5.232667854s so far)
2023-04-12T12:50:04.201 proxy[4d89622c642058] ams [info] waiting for machine to be reachable on 0.0.0.0:8080 (waited 11.236709907s so far)
2023-04-12T12:50:10.205 proxy[4d89622c642058] ams [info] waiting for machine to be reachable on 0.0.0.0:8080 (waited 17.240284097s so far)
2023-04-12T12:50:15.209 proxy[4d89622c642058] ams [error] failed to connect to machine: gave up after 15 attempts (in 22.24421674s)
2023-04-12T12:50:17.217 proxy[4d89622c642058] ams [info] Starting machine
2023-04-12T12:50:17.567 app[4d89622c642058] ams [info] Starting init (commit: e878f33)...

Is this correct? I can’t tell tbh.

# Run LiteFS as the entrypoint. Anything after the double-dash is run as a
# subprocess by LiteFS. This allows the file system to mount and initialize
# before the application starts.
ENTRYPOINT litefs mount -- primenv -dsn /primenv_db/db

pavel · April 12, 2023, 1:32pm

Volume should be mounted to data.dir (/var/lib/litefs if you use the config that you posted).
And your app should open a databased under /litefs directory, for example /litefs/app.db

sandervspl · April 12, 2023, 2:17pm

I just can’t seem to figure it out what I’m doing wrong. I’ve read two example repo’s and all docs but it’s just not working.

This is my Dockerfile:

# Fetch the LiteFS binary using a multi-stage build.
FROM flyio/litefs:0.2 AS litefs

# Install dependencies only when needed
FROM node:16-alpine AS builder
WORKDIR /app
COPY . .

# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat

RUN npm ci

ENV NEXT_TELEMETRY_DISABLED 1

# Add `ARG` instructions below if you need `NEXT_PUBLIC_` variables
# then put the value on your fly.toml
ARG NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=""

RUN npm run build

# If using npm comment out above and use below instead
# RUN npm run build


# Production image, copy all the files and run next
FROM node:16-alpine AS runner
WORKDIR /app

ENV NODE_ENV production
ENV NEXT_TELEMETRY_DISABLED 1

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

COPY --from=builder /app ./

USER nextjs

ADD etc/litefs.yml /etc/litefs.yml

# Ensure our mount & data directories exists before mounting with LiteFS.
# RUN mkdir -p /data /mnt/sqlite

# Run LiteFS as the entrypoint. Anything after the double-dash is run as a
# subprocess by LiteFS. This allows the file system to mount and initialize
# before the application starts.
ENTRYPOINT "litefs"

fly.toml

# fly.toml file generated for primenv on 2023-04-12T15:20:41+02:00

app = "primenv"
kill_signal = "SIGINT"
kill_timeout = 5
primary_region = "ams"

[build]
  [build.args]
    NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY = ""

[env]
  PORT = "8080"

[mounts]
  source = "primenv_db"
  destination = "/mnt/sqlite"

[[services]]
  protocol = "tcp"
  internal_port = 8080
  processes = ["app"]

  [[services.ports]]
    port = 80
    handlers = ["http"]
    force_https = true

  [[services.ports]]
    port = 443
    handlers = ["tls", "http"]
  [services.concurrency]
    type = "connections"
    hard_limit = 25
    soft_limit = 20

litefs.yml

# The path to where the SQLite database will be accessed.
fuse:
  dir: '/primenv_db'

# The path to where the underlying volume mount is.
data:
  dir: '/mnt/sqlite'

# Execute this subprocess once LiteFS connects to the cluster.
exec: 'npm run start --sqlite-path=/primenv_db/db'

lease:
  type: 'static'
  candidate: true

Is anything here obviously wrong? I can’t see it. My server is still crashing non-stop.

pavel · April 12, 2023, 2:22pm

You need to copy litefs binary from flyio/litefs:0.2 image and also install fuse in your final docker image:

sandervspl · April 12, 2023, 2:38pm

Still not working I’ve renamed everything to match the docs for now.

Dockerfile

# Install dependencies only when needed
FROM node:16-alpine AS builder
WORKDIR /app
COPY . .

# Fetch the LiteFS binary using a multi-stage build.
FROM flyio/litefs:0.3 AS litefs

COPY --from=flyio/litefs:0.3 /usr/local/bin/litefs /usr/local/bin/litefs

# Setup our environment to include FUSE & SQLite. We install ca-certificates
# so we can communicate with the Consul server over HTTPS. cURL is added so
# we can call our HTTP endpoints for debugging.
RUN apk add fuse sqlite

# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat

RUN npm ci

ENV NEXT_TELEMETRY_DISABLED 1

# Add `ARG` instructions below if you need `NEXT_PUBLIC_` variables
# then put the value on your fly.toml
ARG NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=""

RUN npm run build

# If using npm comment out above and use below instead
# RUN npm run build


# Production image, copy all the files and run next
FROM node:16-alpine AS runner
WORKDIR /app

ENV NODE_ENV production
ENV NEXT_TELEMETRY_DISABLED 1

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

COPY --from=builder /app ./

USER nextjs

ADD etc/litefs.yml /etc/litefs.yml

# Ensure our mount & data directories exists before mounting with LiteFS.
# RUN mkdir -p /litefs /var/lib/litefs

# Run LiteFS as the entrypoint. Anything after the double-dash is run as a
# subprocess by LiteFS. This allows the file system to mount and initialize
# before the application starts.
ENTRYPOINT litefs -- primenv_db -dsn /litefs/db

fly.toml

# fly.toml file generated for primenv on 2023-04-12T16:31:35+02:00

app = "primenv"
kill_signal = "SIGINT"
kill_timeout = 5
primary_region = "ams"

[build]
  [build.args]
    NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY = ""

[env]
  PORT = "8080"

[mounts]
  source = "litefs"
  destination = "/var/lib/litefs"

[[services]]
  protocol = "tcp"
  internal_port = 8080
  processes = ["app"]

  [[services.ports]]
    port = 80
    handlers = ["http"]
    force_https = true

  [[services.ports]]
    port = 443
    handlers = ["tls", "http"]
  [services.concurrency]
    type = "connections"
    hard_limit = 25
    soft_limit = 20

litefs.yml

# The path to where the SQLite database will be accessed.
fuse:
  dir: '/litefs'

# The path to where the underlying volume mount is.
data:
  dir: '/var/lib/litefs'

# Execute this subprocess once LiteFS connects to the cluster.
exec: 'npm run start --sqlite-path=/litefs/db'

lease:
  type: 'static'
  candidate: true

sandervspl · April 12, 2023, 2:39pm

Is this line correct? What should I put in the place of primenv_db exactly? Now I’m putting the volume name.

ENTRYPOINT litefs -- primenv_db -dsn /litefs/db

pavel · April 12, 2023, 2:48pm

Since you already have exec entry in your litefs config, you can set ENTRYPOINT to litefs mount and that’s it.

This command will start litefs first and later start whatever is specified in exec entry of litefs config.

sandervspl · April 12, 2023, 2:53pm

I changed it to ENTRYPOINT litefs mount. Server is still crashing

pavel · April 12, 2023, 3:07pm

Missed something in your previous message.

You are running COPY --from=flyio/litefs:0.3 /usr/local/bin/litefs /usr/local/bin/litefs and RUN apk add fuse sqlite in a temporary build container, not your final image.

You need to run them in the last node:16-alpine container (runner). That’s the one you actually deploy to Fly. Otherwise, it won’t have fuse and litefs installed.

sandervspl · April 12, 2023, 3:08pm

I’ve tried that. I get permission denied errors if I do that.

 => ERROR [runner 8/8] RUN apk add fuse sqlite                                                                                                                                                                                                  0.3s
------
 > [runner 8/8] RUN apk add fuse sqlite:
#16 0.242 ERROR: Unable to lock database: Permission denied
#16 0.242 ERROR: Failed to open apk database: Permission denied
------
Error failed to fetch an image or build from source: error building: executor failed running [/bin/sh -c apk add fuse sqlite]: exit code: 99

benbjohnson · April 12, 2023, 3:11pm

It might be easier to take out the USER nextjs and run as root to get LiteFS working first. Then you can try re-adding that later. Running FUSE as a non-root user takes some extra steps. Also, it looks like apk doesn’t have permission to run as-is.

sandervspl · April 12, 2023, 3:27pm

I’ve disabled USER nextjs. Server is still crashing.

# Production image, copy all the files and run next
FROM node:16-alpine AS runner
WORKDIR /app

ENV NODE_ENV production
ENV NEXT_TELEMETRY_DISABLED 1

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

COPY --from=builder /app ./

# USER nextjs

ADD etc/litefs.yml /etc/litefs.yml

# Run LiteFS as the entrypoint. Anything after the double-dash is run as a
# subprocess by LiteFS. This allows the file system to mount and initialize
# before the application starts.
ENTRYPOINT litefs mount

benbjohnson · April 12, 2023, 3:42pm

Do you have error messages from the fly logs?

sandervspl · April 12, 2023, 3:46pm

Sure, here they are

sandervspl · April 12, 2023, 5:55pm

I’ve made the repo public so you can take a better look at the setup.

I hope that helps

simse · April 12, 2023, 9:41pm

This line copying the litefs binary: COPY --from=flyio/litefs:0.3 /usr/local/bin/litefs /usr/local/bin/litefs needs to be in your runner stage. You could put it immediately before your ENTRYPOINT.

benbjohnson · April 12, 2023, 10:30pm

Thanks, @simse. That’s correct. I just checked the repo from @sandervspl and it looks like I can deploy the Dockerfile. I tried removing the initial builder section just to isolate the LiteFS stuff and it looks like it runs now.

Another useful tip for debugging is to deploy with:

ENTRYPOINT sleep inf

And then you can SSH in and try to get things running manually to see what’s missing:

$ fly ssh console --pty

Topic		Replies	Views
Issue Deploying Next.js / Prisma app JavaScript postgres , nodejs	13	3296	March 14, 2025
NextJS deploy fails with "Error failed to fetch an image or build from source: executing lifecycle: failed with status code: 51" Build debugging	2	2923	March 6, 2022
Can't deploy a NextJS app to the production Questions / Help	2	360	August 28, 2023
Fly Secrets not populated during build	17	3149	December 7, 2022
Deployment with NextJS + Prisma, getStaticProps fails because Prisma can't reach db proxy Build debugging	4	561	June 27, 2023

Unable to deploy NextJS app with sqlite db

Related topics