PostgreSQL volume not mounted, running database as non-persistent

Hello new programmer here,

I have a problem with my app: it was deployed a couple of weeks ago and all seemed fine. I could reach the app, login and create profiles.

Anyway I’ve noticed these days that the data are non-persistent: every time the machine stops then starts again by making new migrations. I’m running one machine with one volume attached.
From the log:

[info] PostgreSQL config file doesn’t exist, initializing database…

[info] sed: /postgres-volume/run/postgresql/data/postgresql.conf: No such file or directory

[info] PostgreSQL volume not mounted, running database as non-persistent (new deploys erase changes not saved in migrations)

I don’t know how to solve the problem about the config file. Did I do something wrong with the setup?

It may be connected with the local Docker instance, I’m looking it up but any suggestion is welcome!

$ fly doctor --verbose
Testing authentication token… PASSED
Testing flyctl agent… PASSED
Testing local Docker instance… Nope
(We got: failed pinging docker instance: error during connect: this error may indicate that the docker daemon is not running: Get “http://%2F%2F.%2Fpipe%2Fdocker_engine/_ping”: open //./pipe/docker_engine: The system cannot find the file specified.)
This is fine, we’ll use a remote builder.
Pinging WireGuard gateway (give us a sec)… PASSED
Testing WireGuard DNS… PASSED
Testing WireGuard Flaps… PASSED

App specific checks for opentribe:
Checking that app has ip addresses allocated… PASSED
Checking A record for *** … PASSED

Build checks for opentribe:
Checking docker context size (this may take little bit)… PASSED (9.4 MB)
Checking for .dockerignore… PASSED

I’m confused…

$ flyctl postgres config show
Error: command is not compatible with this image

This is indeed pretty atypical… What do fly status, fly m list, and fly apps list show?


The fly.toml and Dockerfile would probably help with the broader volume question, as well…

This while the app was running:

$ fly status
ID              STATE   ROLE    REGION  CHECKS  IMAGE                                           CREATED                 UPDATED
17811e16fe5258  started unknown otp             opentribe:deployment-01HKSNKRBPNAGPXHSA1AJ56MRB 2023-12-21T16:13:35Z    2024-01-11T13:36:59Z
$ fly m list
1 machines have been retrieved from app opentribe.
View them in the UI here (​https://fly.io/apps/opentribe/machines/)

opentribe
ID              NAME            STATE   REGION  IMAGE                                           IP ADDRESS                      VOLUME                  CREATED                 LAST UPDATED            APP PLATFORM    PROCESS GROUP   SIZE
17811e16fe5258  dry-shape-8368  started otp     opentribe:deployment-01HKSNKRBPNAGPXHSA1AJ56MRB fdaa:3:41ab:a7b:1ad:f091:f190:2 vol_9vl18qj1n959o8o4    2023-12-21T16:13:35Z    2024-01-11T13:36:59Z    v2              app             shared-cpu-1x:256MB
$ fly apps list
NAME                                    OWNER           STATUS          PLATFORM        LATEST DEPLOY
fly-builder-weathered-night-4454        personal        suspended       machines
next-js-ecommerce-store                 personal        suspended       machines        Dec 3 2023 17:39
opentribe                               personal        deployed        machines        Jan 10 2024 12:32

And this is my fly.toml:

app = "opentribe"
primary_region = "otp"

# Apps without volume: Comment out the [mounts] configuration below
[mounts]
  source = "pg_data"
  destination = "/pg_data"

[env]
  PORT = "8080"

[http_service]
  internal_port = 8080
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true

And the Dockerfile:

FROM node:lts-alpine AS builder
ENV NODE_ENV production
# Install necessary tools
RUN apk add --no-cache libc6-compat yq --repository=https://dl-cdn.alpinelinux.org/alpine/edge/community
# Install pnpm
RUN corepack enable && corepack prepare pnpm@latest --activate
WORKDIR /app
# Copy the content of the project to the machine
COPY . .
RUN yq --inplace --output-format=json '(.dependencies = .dependencies * (.devDependencies | to_entries | map(select(.key | test("^(typescript|@types/*|eslint-config-upleveled)$"))) | from_entries)) | (.devDependencies = {})' package.json
RUN pnpm install
RUN pnpm build

# Multi-stage builds: runner stage
FROM node:lts-alpine AS runner
ENV NODE_ENV production
# Install necessary tools
RUN apk add bash postgresql
RUN corepack enable && corepack prepare pnpm@latest --activate
WORKDIR /app

# Copy built app
COPY --from=builder /app/.next ./.next

# Copy only necessary files to run the app (minimize production app size, improve performance)
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/migrations ./migrations
COPY --from=builder /app/public ./public
COPY --from=builder /app/package.json ./
COPY --from=builder /app/.env.production ./
COPY --from=builder /app/next.config.js ./

# Copy start script and make it executable
COPY --from=builder /app/scripts ./scripts
RUN chmod +x /app/scripts/fly-io-start.sh

CMD ["./scripts/fly-io-start.sh"]

Thanks for the additional details… It is unusual to meld the Postgres database together in a single machine with the web application, which is why so many things that are sounding easy for other users are turning out otherwise…

Was there a particular reason you needed this way?

(If not, then the path of least resistance would be to instead create a separate Postgres Flex app, via fly pg create.)

I agree with this - typically users will create a separate postgres database with fly pg create.


It looks like your volume is mounted at /pg_data but your database is trying to use /postgres-volume, so the data is not persisted. Only the data in the volume is persisted.

Thanks for your answer!

…and no, no particular reason.
The fact is that it is a project made during a course, but the docker part wasn’t part of it and so we were provided with a file.
After upgrading to PostgreSQL 16 I’m trying to make it work again, and trying to learn something in the process.
One last question about your suggestion:

(If not, then the path of least resistance would be to instead create a separate Postgres Flex app, via fly pg create.)

This would imply to delete the app and create a new one? It would not be a problem anyway, just asking for the simpler solution.

Uh!
Yep, that doesn’t sound optimal… Thanks for making it clear!

Not quite: you would keep the opentribe app, but responsibility for running its database would be transferred to a new, ready-made auxiliary. (The fact that both of these are referred to as “apps” is a bit of a quirk in Fly terminology.)

$ fly pg create --name opentribe-db  # choose Development sizing.
$ fly pg attach opentribe-db -a opentribe

The latter would add a DATABASE_URL secret to your web application, which you would use to connect to the new database over the encrypted internal network.


I don’t run Next.js myself, but the following look like the right instructions:

https://fly.io/docs/js/frameworks/nextjs/#connecting-to-databases


Finally, you would want to remove the Postgres startup code from your fly-io-start.sh—so you don’t get confusing messages in the logs, etc., :dragon:.

And if this later turns out to be a more serious application that people are relying on for reservations, and the like, then you can upgrade to the more reliable High Availability configuration mentioned in the intro.

Hope this helps!

Hey, thanks so much!
I’ll give it a try as soon as I manage.