Fly machine cannot resolve executable path

Questions / Help Build debugging
1

I have an extremely simple Dockerfile and fly.toml (I’ll paste them at the bottom of this post). However, when I try to deploy my image to a Fly machine, the deploy logs show that my start script can’t be found:

05:41:20
ERROR Error: failed to spawn command: /usr/local/bin/start.sh: No such file or directory (os error 2)
05:41:20
    does `/usr/local/bin/start.sh` exist and is it executable?

The start.sh script absolutely does exist - I’ve pulled the image being used down locally (via fly auth docker and fly image show), and poked around in the image. Not only does the start.sh script exist, but when I run the image on my local machine it finds the script and starts up just fine.

This seems like it could be the same thing as Docker image works locally, but not on Fly.io; getting `command not found` - #17 by mboyea ? But I tried the solution from that thread (manually setting PATH in the [[env]] section of fly.toml) and it didn’t resolve the issue.

This is extremely frustrating, I can’t figure out what is going wrong or how to fix it.

Here’s my Dockerfile:

FROM debian:bookworm-slim

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update \
  && apt-get install -y --no-install-recommends \
    bash \
    ca-certificates \
    curl \
    iproute2 \
    iptables \
  && rm -rf /var/lib/apt/lists/*

RUN curl -fsSL https://opencode.ai/install | bash

COPY --from=docker.io/tailscale/tailscale:stable /usr/local/bin/tailscaled /app/tailscaled
COPY --from=docker.io/tailscale/tailscale:stable /usr/local/bin/tailscale /app/tailscale
RUN mkdir -p /var/run/tailscale /var/cache/tailscale /var/lib/tailscale

RUN mkdir -p /data/workspace

WORKDIR /data/workspace

COPY start.sh /usr/local/bin/start.sh
RUN chmod +x /usr/local/bin/start.sh

EXPOSE 4096

ENV PATH="/root/.local/bin:${PATH}"
ENV XDG_DATA_HOME=/data

CMD ["/usr/local/bin/start.sh"]

and my fly.toml:

app = "opencode-nova"
primary_region = "ewr"

[build]
dockerfile = "Dockerfile"

[env]
OPENCODE_PORT = "4096"
XDG_DATA_HOME = "/data"
TAILSCALE_HOSTNAME = "nova"

[[mounts]]
source = "opencode_data"
destination = "/data"

Am I missing something obvious?

2

Hm… This error can be caused by a mangled shebang line (#!), actually, even when the script itself does exist…

What does the top of your start.sh look like?

Also beware of Windows-style line endings creeping in, :dragon:.

4

here’s start.sh:

#!/usr/bin/env bash
set -euo pipefail

mkdir -p /data/workspace /data/tailscale /run/tailscale

tailscaled --state=/data/tailscale/tailscaled.state --socket=/run/tailscale/tailscaled.sock &

authkey="${TS_AUTHKEY:-${TAILSCALE_AUTHKEY:-}}"
if [[ -z "${authkey}" ]]; then
  echo "Missing Tailscale auth key. Set TS_AUTHKEY or TAILSCALE_AUTHKEY." >&2
  exit 1
fi

for i in {1..30}; do
  if tailscale --socket=/run/tailscale/tailscaled.sock status >/dev/null 2>&1; then
    break
  fi
  sleep 1
done

tailscale \
  --socket=/run/tailscale/tailscaled.sock \
  up \
  --authkey="${authkey}" \
  --hostname="${TAILSCALE_HOSTNAME:-opencode}" \
  --accept-dns=false \
  --accept-routes=false

exec opencode web --hostname 0.0.0.0 --port "${OPENCODE_PORT:-4096}"

and the line endings seem ok:

$ file start.sh
start.sh: Bourne-Again shell script text executable, ASCII text
5

Another data point, at one point in an attempt to debug I changed the CMD in the Dockerfile to sleep inf (so the machine would stick around long enough for me to console into it). But I got the same failed to spawn command error for sleep.

1 Like
6

Thanks for the additional details… I would reduce the Dockerfile to the following at this point:

FROM debian:bookworm-slim

CMD ["sleep", "inf"]

And then gradually add lines back in to see which one causes it to start failing.

(Or, if not even that boots, then use the classic “destroy the Machine and then redeploy” debugging technique, :sweat_smile:.)

If it’s the ENV PATH=... that’s the culprit, you can just move that into start.sh itself (as export PATH=...). You gave /root/.local/bin seniority in the search order, which might be interfering somehow.

7

I’ve figured it out - it was the WORKDIR /data/workspace line. I’m not sure why that broke PATH in this way, maybe because /data is the mount point for the volume? At any rate, removing that line and putting a cd /data/workspace in start.sh fixed it.

1 Like