Hello everyone,
I am currently trying to deploy a PostgreSQL application that includes a pgvector extension, as specified in the Dockerfile. I’m not entirely certain if it’s correctly implemented; however, I followed a similar post addressing the same issue.
ERROR: extension "vector" is not available.
Monitoring Logs:
$ fly logs -a streams
Waiting for logs...
2023-12-31T08:33:42.710 runner[e286630ece1168] lax [info] Pulling container image registry.fly.io/streams:deployment-01HJZFS99N7Y2QP7RZKYNPPRPK
2023-12-31T08:33:52.324 runner[e286630ece1168] lax [info] Successfully prepared image registry.fly.io/streams:deployment-01HJZFS99N7Y2QP7RZKYNPPRPK (9.614181245s)
2023-12-31T08:33:55.422 runner[e286630ece1168] lax [info] Configuring firecracker
2023-12-31T08:33:55.697 app[e286630ece1168] lax [info] [ 0.062042] PCI: Fatal: No config space access function found
2023-12-31T08:33:55.990 app[e286630ece1168] lax [info] INFO Starting init (commit: 15238e9)...
2023-12-31T08:33:56.067 app[e286630ece1168] lax [info] INFO Preparing to run: `docker-entrypoint.sh npx prisma migrate dev --name init` as root
2023-12-31T08:33:56.081 app[e286630ece1168] lax [info] INFO [fly api proxy] listening at /.fly/api
2023-12-31T08:33:56.092 app[e286630ece1168] lax [info] 2023/12/31 08:33:56 listening on [fdaa:4:c8fd:a7b:26e:4e62:dd08:2]:22 (DNS: [fdaa::3]:53)
2023-12-31T08:33:58.493 app[e286630ece1168] lax [info] Prisma schema loaded from schema.prisma
2023-12-31T08:33:58.502 app[e286630ece1168] lax [info] Datasource "db": PostgreSQL database "streams", schema "public" at "streams-fly-db.flycast:5432"
2023-12-31T08:34:00.062 app[e286630ece1168] lax [info] Applying migration `20231231083400_init`
2023-12-31T08:34:00.067 app[e286630ece1168] lax [info] Error: P3018
2023-12-31T08:34:00.067 app[e286630ece1168] lax [info] A migration failed to apply. New migrations cannot be applied before the error is recovered from. Read more about how to resolve migration issues in a production database: https://pris.ly/d/migrate-resolve
2023-12-31T08:34:00.067 app[e286630ece1168] lax [info] Migration name: 20231231083400_init
2023-12-31T08:34:00.067 app[e286630ece1168] lax [info] Database error code: 0A000
2023-12-31T08:34:00.067 app[e286630ece1168] lax [info] Database error:
2023-12-31T08:34:00.067 app[e286630ece1168] lax [info] ERROR: extension "vector" is not available
2023-12-31T08:34:00.067 app[e286630ece1168] lax [info] DETAIL: Could not open extension control file "/usr/share/postgresql/15/extension/vector.control": No such file or directory.
2023-12-31T08:34:00.067 app[e286630ece1168] lax [info] HINT: The extension must first be installed on the system where PostgreSQL is running.
2023-12-31T08:34:00.067 app[e286630ece1168] lax [info] DbError { severity: "ERROR", parsed_severity: Some(Error), code: SqlState(E0A000), message: "extension \"vector\" is not available", detail: Some("Could not open extension control file \"/usr/share/postgresql/15/extension/vector.control\": No such file or directory."), hint: Some("The extension must first be installed on the system where PostgreSQL is running."), position: None, where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("extension.c"), line: Some(539), routine: Some("parse_extension_control_file") }
2023-12-31T08:34:00.086 app[e286630ece1168] lax [info] WARN Reaped child process with pid: 350 and signal: SIGTERM, core dumped? false
2023-12-31T08:34:00.086 app[e286630ece1168] lax [info] npm notice
2023-12-31T08:34:00.086 app[e286630ece1168] lax [info] npm notice New major version of npm available! 9.6.7 -> 10.2.5
2023-12-31T08:34:00.086 app[e286630ece1168] lax [info] npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.2.5>
2023-12-31T08:34:00.086 app[e286630ece1168] lax [info] npm notice Run `npm install -g npm@10.2.5` to update!
2023-12-31T08:34:00.087 app[e286630ece1168] lax [info] npm notice
2023-12-31T08:34:01.089 app[e286630ece1168] lax [info] INFO Main child exited normally with code: 1
2023-12-31T08:34:01.090 app[e286630ece1168] lax [info] INFO Starting clean up.
2023-12-31T08:34:01.092 app[e286630ece1168] lax [info] WARN hallpass exited, pid: 314, status: signal: 15 (SIGTERM)
2023-12-31T08:34:01.099 app[e286630ece1168] lax [info] 2023/12/31 08:34:01 listening on [fdaa:4:c8fd:a7b:26e:4e62:dd08:2]:22 (DNS: [fdaa::3]:53)
2023-12-31T08:34:02.094 app[e286630ece1168] lax [info] [ 6.459479] reboot: Restarting system
2023-12-31T08:34:02.238 runner[e286630ece1168] lax [info] machine restart policy set to 'no', not restarting
Terminal (local) Logs
Prisma schema loaded from schema.prisma
Datasource "db": PostgreSQL database "streams", schema "public" at "streams-fly-db.flycast:5432"
Applying migration `20231231083400_init`
Error: P3018
A migration failed to apply. New migrations cannot be applied before the error is recovered from. Read more about how to resolve migration issues in a production database: https://pris.ly/d/migrate-resolve
Migration name: 20231231083400_init
Database error code: 0A000
Database error:
ERROR: extension "vector" is not available
DETAIL: Could not open extension control file "/usr/share/postgresql/15/extension/vector.control": No such file or directory.
HINT: The extension must first be installed on the system where PostgreSQL is running.
DbError { severity: "ERROR", parsed_severity: Some(Error), code: SqlState(E0A000), message: "extension \"vector\" is not available", detail: Some("Could not open extension control file \"/usr/share/postgresql/15/extension/vector.control\": No such file or directory."), hint: Some("The extension must first be installed on the system where PostgreSQL is running."), position: None, where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("extension.c"), line: Some(539), routine: Some("parse_extension_control_file") }
WARN Reaped child process with pid: 350 and signal: SIGTERM, core dumped? false
npm notice
npm notice New major version of npm available! 9.6.7 -> 10.2.5
Dockerfile:
# syntax = docker/dockerfile:1
# Adjust NODE_VERSION as desired
ARG NODE_VERSION=18.17.1
FROM node:${NODE_VERSION}-slim as base
LABEL fly_launch_runtime="Remix/Prisma"
# Remix/Prisma app lives here
WORKDIR /app
# Set production environment
ENV NODE_ENV="production"
# Throw-away build stage to reduce size of final image
FROM base as build
# Install packages needed to build node modules and pgvector
RUN apt-get update -qq && \
apt-get install -y build-essential openssl pkg-config python-is-python3 git postgresql-server-dev-15
# Install node modules
COPY --link .npmrc package.json ./
RUN npm install --include=dev
# Generate Prisma Client
COPY --link prisma .
RUN npx prisma generate
# Copy application code
COPY --link . .
# Build application
RUN npm run build
# Build pgvector
WORKDIR /home
RUN git clone --branch v0.5.0 https://github.com/pgvector/pgvector.git
WORKDIR /home/pgvector
RUN make && make install
# Switch back to app directory
WORKDIR /app
# Remove development dependencies
RUN npm prune --omit=dev
# Final stage for app image
FROM base
# Install packages needed for deployment
RUN apt-get update -qq && \
apt-get install --no-install-recommends -y openssl && \
rm -rf /var/lib/apt/lists /var/cache/apt/archives
# Copy built application
COPY --from=build /app /app
# Start the server by default, this can be overwritten at runtime
EXPOSE 3000
CMD [ "npm", "run", "start" ]
Also, some insights from GPT-4
-
Separate Concerns: Ensure that pgvector is installed in the same PostgreSQL instance that your application connects to. Installing pgvector in the Docker image is not sufficient if the PostgreSQL database is hosted elsewhere (like on Fly.io’s PostgreSQL service).
-
Install pgvector on PostgreSQL Server: You need to install pgvector on the PostgreSQL server. This might require customizing the PostgreSQL instance or using a PostgreSQL server where you have the privileges to install extensions. .
-
Consider Using a Custom PostgreSQL Image: If you have control over the PostgreSQL instance (like hosting it yourself), you could create a custom Docker image for PostgreSQL with pgvector pre-installed. Then, deploy this customized PostgreSQL instance to Fly.io or your preferred host.
Any suggestion to possible solutions, or is there additional information I could provide?
Thank you!