Can't deploy suddenly

So i wanted to update my bot and it started to give me Error: UnhandledIoError(Os { code: 2, kind: NotFound, message: "No such file or directory" }) whenever i try to deploy

My docker file:

# using ubuntu LTS version
FROM ubuntu:20.04 AS builder-image

# avoid stuck build due to user prompt
ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get install --no-install-recommends -y python3.9 python3.9-dev python3.9-venv python3-pip python3-wheel build-essential && \
	apt-get clean && rm -rf /var/lib/apt/lists/*

# create and activate virtual environment
# using final folder name to avoid path issues with packages
RUN python3.9 -m venv /home/kexobot/venv
ENV PATH="/home/kexobot/venv/bin:$PATH"

# install requirements
COPY requirements.txt .
RUN pip3 install --no-cache-dir wheel
RUN pip3 install --no-cache-dir -r requirements.txt

FROM ubuntu:20.04 AS runner-image
RUN apt-get update && apt-get install --no-install-recommends -y python3.9 python3-venv && \
    # Install ffmpeg and opus
    apt-get install -y ffmpeg && apt-get install libopus0 && \
    # We need wget to set up the PPA and xvfb and gnupg to have a virtual screen and unzip to install the Chromedriver
    apt-get install -y wget xvfb unzip && apt-get install gnupg -y  && \
    # Set up the Chrome PPA
    wget -q -O - | apt-key add - && \
    echo "deb stable main" >> /etc/apt/sources.list.d/google.list && \
    # Set up the Chrome PPA
    apt-get update && apt-get install -y google-chrome-stable && \
	apt-get clean && rm -rf /var/lib/apt/lists/*

# Set up Chromedriver Environment variables

# Download and install Chromedriver

# Put Chromedriver into the PATH

RUN useradd --create-home kexobot
COPY --from=builder-image /home/kexobot/venv /home/kexobot/venv

USER kexobot
RUN mkdir /home/kexobot/code
WORKDIR /home/kexobot/code
COPY . .


# make sure all messages always reach console

# activate virtual environment
ENV VIRTUAL_ENV=/home/kexobot/venv
ENV PATH="/home/kexobot/venv/bin:$PATH"

# /dev/shm is mapped to shared memory and should be used for gunicorn heartbeat
# this will improve performance and avoid random freezes
CMD ["gunicorn","-b", "", "-w", "4", "-k", "gevent", "--worker-tmp-dir", "/dev/shm", "app:app", "python", "python"]

My fly.toml file:

# fly.toml file generated for kexobot on 2022-08-28T17:35:22+02:00

app = "kexobot"
kill_signal = "SIGINT"
kill_timeout = 5

  web = ""
  worker = ""

  allowed_public_ports = []
  auto_rollback = true

  http_checks = []
  internal_port = 8080
  processes = ["app"]
  protocol = "tcp"
  script_checks = []
    hard_limit = 25
    soft_limit = 20
    type = "connections"

    force_https = true
    handlers = ["http"]
    port = 80

    handlers = ["tls", "http"]
    port = 443

    grace_period = "1s"
    interval = "15s"
    restart_limit = 0
    timeout = "2s"

Remove these 3 lines from your fly.toml if you do not intend to use multi-process (you probably want the CMD entrypoint in your dockerfile to run instead?):

- [processes]
-   web = ""
-   worker = ""
  http_checks = []
  internal_port = 8080
-  processes = ["app"]

Also, append --local-only to flyctl deploy in case the runc (docker) image could not be built remotely by Fly’s builders (unlikely to be the case).

EXPOSE directive in your dockerfile won’t be honoured by Fly. The equivalent fly.toml entries are port (public; which Fly’s load-balancer listens on for tcp/udp/http, on behalf of your Fly app) and internal_port (the one your process listens on, to which Fly’s load balancer forwards udp-packets/tcp-connections/http-requests to). More: App Configuration (fly.toml) · Fly Docs

I removed all 4 processes lines from fly.toml, and also i removed EXPOSE command from dockerfile, then i launched fly deploy --local-only command, i still get same error:

(here’s full error code that repeats 3 times)

2022-08-29T07:53:49.168 runner[1c9754e7] fra [info] Starting instance

2022-08-29T07:53:50.848 runner[1c9754e7] fra [info] Configuring virtual machine

2022-08-29T07:53:50.855 runner[1c9754e7] fra [info] Pulling container image

2022-08-29T07:54:04.799 runner[1c9754e7] fra [info] Unpacking image

2022-08-29T07:54:20.527 runner[1c9754e7] fra [info] Preparing kernel init

2022-08-29T07:54:20.939 runner[1c9754e7] fra [info] Configuring firecracker

2022-08-29T07:54:21.040 runner[1c9754e7] fra [info] Starting virtual machine

2022-08-29T07:54:21.352 app[1c9754e7] fra [info] Starting init (commit: f815f49)...

2022-08-29T07:54:21.383 app[1c9754e7] fra [info] Preparing to run: `gunicorn -b -w 4 -k gevent --worker-tmp-dir /dev/shm app:app python python` as yourdir

2022-08-29T07:54:21.400 app[1c9754e7] fra [info] Error: UnhandledIoError(Os { code: 2, kind: NotFound, message: "No such file or directory" })

2022-08-29T07:54:21.401 app[1c9754e7] fra [info] [ 0.178081] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100

2022-08-29T07:54:21.401 app[1c9754e7] fra [info] [ 0.179120] CPU: 0 PID: 1 Comm: init Not tainted 5.12.2 #1

2022-08-29T07:54:21.402 app[1c9754e7] fra [info] [ 0.179818] Call Trace:

2022-08-29T07:54:21.402 app[1c9754e7] fra [info] [ 0.180102] show_stack+0x52/0x58

2022-08-29T07:54:21.402 app[1c9754e7] fra [info] [ 0.180493] dump_stack+0x6b/0x86

2022-08-29T07:54:21.403 app[1c9754e7] fra [info] [ 0.180888] panic+0xfb/0x2bc

2022-08-29T07:54:21.403 app[1c9754e7] fra [info] [ 0.181257] do_exit.cold+0x60/0xb0

2022-08-29T07:54:21.404 app[1c9754e7] fra [info] [ 0.181803] do_group_exit+0x3b/0xb0

2022-08-29T07:54:21.404 app[1c9754e7] fra [info] [ 0.182311] __x64_sys_exit_group+0x18/0x20

2022-08-29T07:54:21.405 app[1c9754e7] fra [info] [ 0.183146] entry_SYll_64+0x38/0x50

2022-08-29T07:54:21.405 app[1c9754e7] fra [info] [ 0.183146] entry_SYSCALL_64_after_hwframe+0x44/0xae

2022-08-29T07:54:21.406 app[1c9754e7] fra [info] [ 0.183626] RIP: 0033:0x7efff297ceb9

2022-08-29T07:54:21.407 app[1c9754e7] fra [info] [ 0.184026] Code: eb ef 48 8b 76 28 e9 a5 03 00 00 64 48 8b 04 25 00 00 00 00 48 8b b0 b0 00 00 00 e9 af ff ff ff 48 63 ff b8 e7 00 00 00 0f 05 <ba> 3c 00 00 00 48 89 d0 0f 05 eb f9 66 2e 0f 1f 84 00 00 00 00 00

2022-08-29T07:54:21.408 app[1c9754e7] fra [info] [ 0.185903] RSP: 002b:00007ffd91b006a8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7

2022-08-29T07:54:21.409 app[1c9754e7] fra [info] [ 0.186754] RAX: ffffffffffffffda RBX: 00007efff27e2180 RCX: 00007efff297ceb9

2022-08-29T07:54:21.410 app[1c9754e7] fra [info] [ 0.187586] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000001

2022-08-29T07:54:21.411 app[1c9754e7] fra [info] [ 0.188319] RBP: 0000000000000001 R08: 00007efff2a50c08 R09: 0000000000000000

2022-08-29T07:54:21.411 app[1c9754e7] fra [info] [ 0.189808] R10: 0000000000000000 R11: 0000000000000246 R12: 00007ffd91b00708

2022-08-29T07:54:21.412 app[1c9754e7] fra [info] [ 0.189808] R13: 00007ffd91b00718 R14: 0000000000000000 R15: 0000000000000000

2022-08-29T07:54:21.412 app[1c9754e7] fra [info] [ 0.190610] Kernel Offset: disabled

2022-08-29T07:54:21.413 app[1c9754e7] fra [info] [ 0.190977] Rebooting in 1 seconds..

I don’t think the issue you’re seeing is specific to Fly.

It could be that gunicorn isn’t in PATH? If so, RUN which gunicorn should help figure that part out.

I haven’t used gunicorn myself, but the documentation has no note about the option python <> which you do use in CMD (as: "python", "python").

Finally, could enable debug logging with gunicorn by adding --log-level, debug to dockerfile CMD?

Now i get this error:

ModuleNotFoundError: No module named '--log-level debug'

I still don’t get that it worked perfectly yesterday, but something happened and it just gave me only errors.

Hi @Kexo

Does your dockerfile work if you run it locally on your computer using a fresh clone of your repository?

Ok so i decided to just have one procces running classic way: CMD ["python", ""]
and second procces will be on another server.

1 Like