fuse device not found, try 'modprobe fuse' first

When trying to deploy an app using litefs with flyctl deploy I get an error saying primary lease acquired, advertising as http://truffle-plays:20202 #12 0.266 mount helper error: fusermount: fuse device not found, try 'modprobe fuse' first #12 0.267 ERROR: cannot init file system: cannot open file system: fusermount: exit status 1
I have looked at this post LiteFS example not working locally (fuse device not found) - #3 by cup but im just trying to deploy it right now. I install fuse in the Dockerfile before it runs litefs mount

Do you have a Dockerfile you can share?

Dockerfile:


# Make a fake Rust app to keep a cached layer of compiled crates
RUN USER=root cargo new app
WORKDIR /usr/src/app
COPY Cargo.toml Cargo.lock ./
# Needs at least a main.rs file with a main function
RUN mkdir src && echo "fn main(){}" > src/main.rs
# Will build all dependent crates in release mode
RUN --mount=type=cache,target=/usr/local/cargo/registry \
  --mount=type=cache,target=/usr/src/app/target \
  cargo build --release

# Copy the rest
COPY . .
# Build (install) the actual binaries
RUN --mount=type=cache,target=/usr/local/cargo/registry \
  --mount=type=cache,target=/usr/src/app/target \
  cargo install --path .

# Runtime image
FROM debian:bullseye-slim

# Run as "app" user
RUN useradd -ms /bin/bash app

USER app
WORKDIR /app

FROM debian:buster-slim

# Download latest listing of available packages:
RUN apt-get -y update
# Upgrade already installed packages:
RUN apt-get -y upgrade
# Install a new package:
RUN apt-get -y install fuse sqlite

COPY --from=flyio/litefs:0.3 /usr/local/bin/litefs /usr/local/bin/litefs
COPY ./etc/litefs.yml ./litefs.yml
RUN litefs mount

# Get compiled binaries from builder's cargo install directory
COPY --from=builder /usr/local/cargo/bin/truffle-plays-server /app/truffle-plays-server

# No CMD or ENTRYPOINT, see fly.toml with `cmd` override.

litefs.yml:

fuse:
  dir: "/litefs"

# This directory is where LiteFS will store internal data.
# You must place this directory on a persistent volume.
data:
  dir: "/var/lib/litefs"

lease:
  type: "static"
  candidate: true
  hostname: "truffle-plays"
  advertise-url: "http://truffle-plays:20202"

I think the issue is running it as a non-root user. I have a change to allow for running as a non-root user (#251) but it’ll be released in the next version (v0.4.0).

I’m still running into this problem in 0.4, any thoughts? I pushed up a draft PR to my repo here: First attempt at Litefs support by jakeprem · Pull Request #5 · jakeprem/taxon · GitHub

Hoping this repo can serve as a example of setting up Elixir to use Litefs.

Or here are the relevant files.
fuse.conf:

user_allow_other

litefs.yml:

fuse:
  dir: "/litefs"
  allow-other: true

data:
  dir: "/var/lib/litefs"

# exec:
#   - cmd: "/app/bin/server"

lease:
  type: "consul"
  candidate: ${FLY_REGION == PRIMARY_REGION}
  promote: true
  advertise-url: "http://${FLY_ALLOC_ID}.vm.${FLY_APP_NAME}.internal.20202"
  consul:
    url: "${FLY_CONSUL_URL}"
    key: "#{FLY_APP_NAME}/primary"
  # type: "static"
  # candidate: true

Dockerfile:

# Find eligible builder and runner images on Docker Hub. We use Ubuntu/Debian
# instead of Alpine to avoid DNS resolution issues in production.
#
# https://hub.docker.com/r/hexpm/elixir/tags?page=1&name=ubuntu
# https://hub.docker.com/_/ubuntu?tab=tags
#
# This file is based on these images:
#
#   - https://hub.docker.com/r/hexpm/elixir/tags - for the build image
#   - https://hub.docker.com/_/debian?tab=tags&page=1&name=bullseye-20220801-slim - for the release image
#   - https://pkgs.org/ - resource for finding needed packages
#   - Ex: hexpm/elixir:1.14.0-erlang-25.1-debian-bullseye-20220801-slim
#
ARG ELIXIR_VERSION=1.14.0
ARG OTP_VERSION=25.1
ARG DEBIAN_VERSION=bullseye-20220801-slim

ARG BUILDER_IMAGE="hexpm/elixir:${ELIXIR_VERSION}-erlang-${OTP_VERSION}-debian-${DEBIAN_VERSION}"
ARG RUNNER_IMAGE="debian:${DEBIAN_VERSION}"

FROM ${BUILDER_IMAGE} as builder

# install build dependencies
RUN apt-get update -y && apt-get install -y build-essential git fuse3 \
    && apt-get clean && rm -f /var/lib/apt/lists/*_*

# prepare build dir
WORKDIR /app

# install hex + rebar
RUN mix local.hex --force && \
    mix local.rebar --force

# set build ENV
ENV MIX_ENV="prod"

COPY ./litefs.yml ./
COPY ./fuse.conf /etc/fuse.conf

# install mix dependencies
COPY mix.exs mix.lock ./
RUN mix deps.get --only $MIX_ENV
RUN mkdir config

# copy compile-time config files before we compile dependencies
# to ensure any relevant config change will trigger the dependencies
# to be re-compiled.
COPY config/config.exs config/${MIX_ENV}.exs config/
RUN mix deps.compile

COPY priv priv

COPY lib lib

COPY assets assets

# compile assets
RUN mix assets.deploy

# Compile the release
RUN mix compile

# Changes to config/runtime.exs don't require recompiling the code
COPY config/runtime.exs config/

COPY rel rel
RUN mix release

# start a new build stage so that the final image will only contain
# the compiled release and other runtime necessities
FROM ${RUNNER_IMAGE}

RUN mkdir /litefs
RUN chown -R nobody:root /litefs
RUN mkdir /var/lib/litefs
RUN chown -R nobody:root /var/lib/litefs

RUN apt-get update -y && apt-get install -y libstdc++6 openssl libncurses5 locales fuse3 sqlite3 ca-certificates curl kmod \
  && apt-get clean && rm -f /var/lib/apt/lists/*_*

COPY --from=flyio/litefs:main /usr/local/bin/litefs /usr/bin/litefs
COPY --from=builder --chown=nobody:root /app/litefs.yml /etc/litefs.yml
COPY --from=builder --chown=nobody:root /etc/fuse.conf /etc/fuse.conf

# Set the locale
RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen

ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8

WORKDIR "/app"
RUN chown nobody /app

# set runner ENV
ENV MIX_ENV="prod"

# Only copy the final release from the build stage
COPY --from=builder --chown=nobody:root /app/_build/${MIX_ENV}/rel/taxon ./

USER nobody

# CMD ["/app/bin/server"]
ENTRYPOINT litefs mount

# Appended by flyctl
ENV ECTO_IPV6 true
ENV ERL_AFLAGS "-proto_dist inet6_tcp"

There’s some issues with running with the Docker USER command when using LiteFS. I wrote up an explanation for a possible workaround here although it’s a little hacky: LiteFS v0.4.0 Released - #2 by tj1

We are going to release a SQLite VFS version of LiteFS and that’ll make it easier to avoid these permissions issues in the future.