Fails to load SQLite 3 NIF by Exqlite

greenfork · June 15, 2022, 5:07pm

I’m trying to deploy an almost barebones Phoenix application as an umbrella project with SQLite database. I failed to deploy it via fly launch and followed this instruction on how to deploy it. I created and mounted a volume following the guide just in case, set DATABASE_PATH to point to this mounted volume, still same error.

2022-06-15T16:59:42Z   [info] {:load_failed,
2022-06-15T16:59:42Z   [info]  'Failed to load NIF library: \'/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33\' not found (required by /workspace/_build/prod/lib/exqlite/priv/sqlite3_nif.so)\''}}
2022-06-15T16:59:42Z   [info]16:59:42.892 [error] GenServer #PID<0.537.0> terminating
2022-06-15T16:59:42Z   [info]** (RuntimeError) connect raised UndefinedFunctionError exception. The exception details are hidden, as they may contain sensitive data such as database credentials. You may set :show_sensitive_data_on_connection_error to true when starting your connection if you wish to see all of the details
2022-06-15T16:59:42Z   [info]16:59:42.903 [error] GenServer #PID<0.553.0> terminating
2022-06-15T16:59:42Z   [info]** (RuntimeError) connect raised UndefinedFunctionError exception. The exception details are hidden, as they may contain sensitive data such as database credentials. You may set :show_sensitive_data_on_connection_error to true when starting your connection if you wish to see all of the details
2022-06-15T16:59:42Z   [info]16:59:42.917 [error] GenServer #PID<0.566.0> terminating
2022-06-15T16:59:42Z   [info]** (RuntimeError) connect raised UndefinedFunctionError exception. The exception details are hidden, as they may contain sensitive data such as database credentials. You may set :show_sensitive_data_on_connection_error to true when starting your connection if you wish to see all of the details
2022-06-15T16:59:42Z   [info]    (exqlite 0.11.2) Exqlite.Sqlite3NIF.open/1
2022-06-15T16:59:43Z   [info]16:59:42.939 [error] GenServer #PID<0.572.0> terminating
2022-06-15T16:59:43Z   [info]** (RuntimeError) connect raised UndefinedFunctionError exception. The exception details are hidden, as they may contain sensitive data such as database credentials. You may set :show_sensitive_data_on_connection_error to true when starting your connection if you wish to see all of the details
2022-06-15T16:59:43Z   [info]    (exqlite 0.11.2) Exqlite.Sqlite3NIF.open/1
2022-06-15T16:59:43Z   [info]    (exqlite 0.11.2) lib/exqlite/connection.ex:453: Exqlite.Connection.do_connect/2
2022-06-15T16:59:43Z   [info]    (db_connection 2.4.2) lib/db_connection/connection.ex:82: DBConnection.Connection.connect/2
2022-06-15T16:59:43Z   [info]    (connection 1.1.0) lib/connection.ex:622: Connection.enter_connect/5
2022-06-15T16:59:43Z   [info]    (stdlib 3.15) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
2022-06-15T16:59:43Z   [info]Last message: nil
2022-06-15T16:59:43Z   [info]16:59:42.939 [warning] The on_load function for module Elixir.Exqlite.Sqlite3NIF returned:
2022-06-15T16:59:43Z   [info]{:error,
2022-06-15T16:59:43Z   [info] {:load_failed,
2022-06-15T16:59:43Z   [info]  'Failed to load NIF library: \'/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33\' not found (required by /workspace/_build/prod/lib/exqlite/priv/sqlite3_nif.so)\''}}
2022-06-15T16:59:43Z   [info]16:59:43.016 [notice] Application wind_world exited: shutdown
2022-06-15T16:59:43Z   [info][os_mon] memory supervisor port (memsup): Erlang has closed
2022-06-15T16:59:43Z   [info][os_mon] cpu supervisor port (cpu_sup): Erlang has closed
2022-06-15T16:59:44Z   [info]KeKernel pid terminated",application_controller,"{application_terminated,wind_world,shutdown}"}
2022-06-15T16:59:44Z   [info]Kernel pid terminated (application_controller) ({application_terminated,wind_world,shutdown})
2022-06-15T16:59:44Z   [info]Crash dump is being written to: erl_crash.dump...done
2022-06-15T16:59:45Z   [info]Main child exited normally with code: 1
2022-06-15T16:59:45Z   [info]Starting clean up.
2022-06-15T16:59:45Z   [info]Umounting /dev/vdc from /data

Mark · June 15, 2022, 8:57pm

Hi @greenfork!

From line 2 of your error message, it appears that a system dependency may be missing that’s needed at runtime.

GLIBC_2.33

I haven’t tried doing what you’re doing here, so I can’t say for sure, but it leads me to think it’s probably a missing Linux dependency.

greenfork · June 16, 2022, 1:30am

Would you have any tips on how I could approach this problem? The second line

Failed to load NIF library: ‘/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33’ not found (required by /workspace/_build/prod/lib/exqlite/priv/sqlite3_nif.so)’

says that it couldn’t load the C library at run time. C library is always present on any POSIX system, so I assume that the version is incorrect. Could it be that C library has different versions during build time and during run time?

Another idea, maybe fly deploy using “buildpack” method pushes all my compiled C dependencies to the fly.io? So they were compiled on my machine but try to use the fly’s C library. I have them in /deps/ directory which is ignored in my Mercurial .hgignore. Does fly rely on git?

tj1 · June 16, 2022, 6:15am

Well, it is stating that sqlite3_nif.so is linking to libc.so.6, but it’s the wrong version. Did you compile the project inside the docker image or did you compile it on your machine?

A mismatch has to happen somehow for this error to occur, it’s just a question of tracking down where.

greenfork · June 16, 2022, 6:35am

I followed this guide on deploying Elixir application. So yes, I didn’t have any docker set up involved. I assume that PostgreSQL installation doesn’t require any NIFs? Otherwise everyone would hit the same error as me with SQLite.

Currently I’m working on a docker solution, will post here if it’s successful.

tj1 · June 16, 2022, 7:11am

Not sure about the guides…postgrex is native elixir, so it won’t have any issues.

Also, I would ensure that the release is compiled inside the docker image, otherwise, you may end up with an ERTS mismatch.

greenfork · June 17, 2022, 2:28pm

I solved my problem by using docker to deploy. Haven’t tried to use anything beyond http but at least it works.

Here are my files:

Dockerfile:

ARG ELIXIR_VERSION=1.13.2
ARG OTP_VERSION=24.3.2
ARG DEBIAN_VERSION=bullseye-20210902-slim

ARG BUILDER_IMAGE="hexpm/elixir:${ELIXIR_VERSION}-erlang-${OTP_VERSION}-debian-${DEBIAN_VERSION}"
ARG RUNNER_IMAGE="debian:${DEBIAN_VERSION}"

FROM ${BUILDER_IMAGE} as builder

# install build dependencies
RUN apt-get update -y && apt-get install -y build-essential \
    && apt-get clean && rm -f /var/lib/apt/lists/*_*

# prepare build dir
WORKDIR /app

# install hex + rebar
RUN mix local.hex --force && \
    mix local.rebar --force

# set build ENV
ENV MIX_ENV="prod"

# install mix dependencies
RUN mkdir apps/
RUN mkdir apps/wind_world/
RUN mkdir apps/wind_world_web/
COPY apps/wind_world/mix.exs ./apps/wind_world/
COPY apps/wind_world_web/mix.exs ./apps/wind_world_web/
COPY mix.exs mix.lock ./
RUN mix deps.get --only $MIX_ENV
RUN mkdir config

# copy compile-time config files before we compile dependencies
# to ensure any relevant config change will trigger the dependencies
# to be re-compiled.
COPY config/config.exs config/${MIX_ENV}.exs config/
RUN mix deps.compile

COPY apps/wind_world/lib/ apps/wind_world/lib/
COPY apps/wind_world/priv/ apps/wind_world/priv/

COPY apps/wind_world_web/lib/ apps/wind_world_web/lib/
COPY apps/wind_world_web/priv/ apps/wind_world_web/priv/
COPY apps/wind_world_web/assets/ apps/wind_world_web/assets/

# compile assets
RUN mix cmd --app wind_world_web mix assets.deploy

# Compile the release
RUN mix compile

# Changes to config/runtime.exs don't require recompiling the code
COPY config/runtime.exs config/

COPY rel rel
RUN mix release

# start a new build stage so that the final image will only contain
# the compiled release and other runtime necessities
FROM ${RUNNER_IMAGE}

RUN apt-get update -y && apt-get install -y libstdc++6 openssl libncurses5 locales \
  && apt-get clean && rm -f /var/lib/apt/lists/*_*

# Set the locale
RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen

ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8

WORKDIR /app
RUN chown nobody /app

# set runner ENV
ENV MIX_ENV="prod"

ARG DATABASE_PATH
ENV DATABASE_PATH=${DATABASE_PATH}
ARG SECRET_KEY_BASE
ENV SECRET_KEY_BASE=${SECRET_KEY_BASE}
ARG PORT=4000
ENV PORT=${PORT}

# Only copy the final release from the build stage
COPY --from=builder --chown=nobody:root /app/_build/${MIX_ENV}/rel/wind_world_umbrella ./

USER nobody

CMD ["/app/bin/server"]

.dockerignore:

.git
!.git/HEAD
!.git/refs

.hg

# Common development/test artifacts
**/cover/
**/doc/
**/test/
**/tmp/
**/.elixir_ls

# Mix artifacts
**/_build/
**/deps/
**/*.ez

# Database
*.db
*.db-*

# Generated on crash by the VM
**/erl_crash.dump

# Static artifacts - These should be fetched and built inside the Docker image
apps/wind_world_web/assets/node_modules/
apps/wind_world_web/priv/static/assets/
apps/wind_world_web/priv/static/cache_manifest.json

# My personal ignores
tmpassets/

Topic		Replies	Views
Migration in sqlite3 on volume fails Phoenix elixir , phoenix-files	5	1296	January 27, 2022
Can't deploy Phoenix application due to database not found (/app/bin/migrate) Phoenix postgres	5	1948	June 14, 2022
Elixir SQLITE LiteFS [solved #1] Phoenix elixir , sqlite , litefs	2	730	November 5, 2023
Failing to deploy Phoenix speedrun Questions / Help elixir	3	716	April 5, 2022
New Phoenix app fails on first deploy: ERROR [internal] load build context Phoenix elixir	1	422	October 20, 2023

Fails to load SQLite 3 NIF by Exqlite

Related topics