"syntax error: unexpected word" error when trying to deploy custom Directus Dockerfile

MattClegg · August 20, 2023, 6:14am

I am trying to deploy a Directus Dockerfile I wrote. This Dockerfile copys and builds some custom extensions I’ve written and also applies any schema snapshots.

Here’s the Dockerfile:

FROM node:18-alpine AS builder

WORKDIR /builder

# COPY start.sh ./
COPY package.json package-lock.json ./
COPY extensions/ extensions/
COPY snapshots/ snapshots/

RUN npm ci && npm run build

FROM directus/directus:10.5

USER node
WORKDIR /directus

COPY --from=builder /builder/snapshots ./snapshots
COPY --from=builder /builder/extensions/directus-extension-foo ./extensions/directus-extension-foo
COPY --from=builder /builder/extensions/directus-extension-bar./extensions/directus-extension-bar

CMD npx directus bootstrap && \
    npx directus schema apply snapshots/latest.yml --yes && \
    npx directus start

My fly.toml looks like this:

app = "directus-example"
primary_region = "lhr"

[env]
  DB_CLIENT="sqlite3"
  DB_FILENAME="/data/database/data.db"
  STORAGE_LOCATIONS="local"
  STORAGE_LOCAL_DRIVER="local"
  STORAGE_LOCAL_ROOT="/data"
  PUBLIC_URL="https://directus-example.fly.dev"
  PORT=8080

[build]
  dockerfile = ".\\Dockerfile"

[mounts]
  source="directus_example_data"
  destination="/data"

[http_service]
  internal_port = 8055
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 1
  [http_service.concurrency]
    type = "requests"
    hard_limit = 250
    soft_limit = 200

[checks]
  [checks.admin]
    port = 8055
    type = "http"
    interval = "5s"
    timeout = "10s"
    grace_period = "30s"
    method = "get"
    path = "/admin"

However, when I deploy this app, it fails and the logs look like this. The main error I can see is /bin/sh: line 3: syntax error: unexpected word (expecting ")") but I’m not sure if that’s a red herring.

 Waiting for logs...
2023-08-19T19:48:26.518 app[9185e53f1ee948] lhr [info] INFO [fly api proxy] listening at /.fly/api
2023-08-19T19:48:26.521 app[9185e53f1ee948] lhr [info] 2023/08/19 19:48:26 listening on [fdaa:2:5f23:a7b:2808:8ddf:75a9:2]:22 (DNS: [fdaa::3]:53)
2023-08-19T19:48:26.524 app[9185e53f1ee948] lhr [info] /bin/sh: line 3: syntax error: unexpected word (expecting ")")
2023-08-19T19:48:27.518 app[9185e53f1ee948] lhr [info] INFO Main child exited normally with code: 2
2023-08-19T19:48:27.519 app[9185e53f1ee948] lhr [info] INFO Starting clean up.
2023-08-19T19:48:27.520 app[9185e53f1ee948] lhr [info] INFO Umounting /dev/vdb from /data
2023-08-19T19:48:27.522 app[9185e53f1ee948] lhr [info] WARN hallpass exited, pid: 264, status: signal: 15 (SIGTERM)
2023-08-19T19:48:27.527 app[9185e53f1ee948] lhr [info] 2023/08/19 19:48:27 listening on [fdaa:2:5f23:a7b:2808:8ddf:75a9:2]:22 (DNS: [fdaa::3]:53)
2023-08-19T19:48:28.523 app[9185e53f1ee948] lhr [info] [ 2.340527] reboot: Restarting system
2023-08-19T19:48:28.639 runner[9185e53f1ee948] lhr [info] machine did not have a restart policy, defaulting to restart
2023-08-19T19:48:29.759 app[9185e53f1ee948] lhr [info] [ 0.045014] PCI: Fatal: No config space access function found
2023-08-19T19:48:29.983 app[9185e53f1ee948] lhr [info] INFO Starting init (commit: b437b5b)...
2023-08-19T19:48:30.006 app[9185e53f1ee948] lhr [info] INFO Mounting /dev/vdb at /data w/ uid: 1000, gid: 1000 and chmod 0755
2023-08-19T19:48:30.009 app[9185e53f1ee948] lhr [info] INFO Resized /data to 1069547520 bytes
2023-08-19T19:48:30.010 app[9185e53f1ee948] lhr [info] INFO Preparing to run: `sh /bin/sh -c npx directus bootstrap && npx directus schema apply snapshots/latest.yml --yes && npx directus start` as node
2023-08-19T19:48:30.023 app[9185e53f1ee948] lhr [info] INFO [fly api proxy] listening at /.fly/api
2023-08-19T19:48:30.027 app[9185e53f1ee948] lhr [info] 2023/08/19 19:48:30 listening on [fdaa:2:5f23:a7b:2808:8ddf:75a9:2]:22 (DNS: [fdaa::3]:53)
2023-08-19T19:48:30.031 app[9185e53f1ee948] lhr [info] /bin/sh: line 3: syntax error: unexpected word (expecting ")")
2023-08-19T19:48:31.024 app[9185e53f1ee948] lhr [info] INFO Main child exited normally with code: 2
2023-08-19T19:48:31.024 app[9185e53f1ee948] lhr [info] INFO Starting clean up.

I’ve tested running the Dockerfile locally and it does work. I’ve also bumped the memory usage up to 512mb. Any ideas on why this app can’t start?

rubys · August 20, 2023, 11:20am

what’s in start.sh?

fly.io doesn’t actually run Docker, what it does is use Docker to build OCI images, and at deploy time a VM is created, loaded from the OCI image, and then the ENTRYPOINT/CMD is run.

I suspect that the way the CMD is interpreted is different, and doesn’t handle shell things like && and |. I’d suggest putting the startup commands into a shell script (perhaps start.sh?) and change the command to run that.

MattClegg · August 21, 2023, 6:03am

The start.sh file was an alternative way of starting Directus, but I was running into problems with it as well.

#!/bin/sh

set -ex
mkdir -p /snapshots
mkdir -p /data/database
mkdir -p /data/uploads
mkdir -p /data/extensions
chmod -Rf 777 /snapshots
chmod -Rf 777 /data/database
chmod -Rf 777 /data/uploads
chmod -Rf 777 /data/extensions

npx directus bootstrap
npx directus schema apply snapshots/latest.yml --yes &&
npx directus start

Mainly I was getting errors like:

start.sh: line 1: #!/bin/sh: No such file or directory
+ mkdir -p /snapshots
mkdir: can't create directory '/snapshots': Permission denied

I tried changing the shebang to different variations but each one was failing.

This setup was an attempt to use the scripts from my previous post: min_machines_running = 1 not working correctly - #8 by andie That minimal setup seemed to work, but this one does not.

MattClegg · August 21, 2023, 6:32am

I’ve made some changes and modified the start .sh script, I also removed the shebang as I noticed it would spit out an error but still continue with the script execution.

npx directus bootstrap
npx directus schema apply snapshots/latest.yml --yes
npx directus start

The Dockerfile looks like this:

FROM node:18-alpine AS builder

WORKDIR /builder

COPY ./newstart.sh ./newstart.sh
COPY package.json package-lock.json ./
COPY extensions/ extensions/
COPY snapshots/ snapshots/

RUN npm ci && npm run build

FROM directus/directus:10.5
WORKDIR /directus

# probably shouldn't use root here but it works for now
USER root
RUN apk add --no-cache bash

COPY --from=builder /builder/newstart.sh ./newstart.sh
RUN chmod +x ./newstart.sh

USER node

COPY --from=builder /builder/snapshots ./snapshots
COPY --from=builder /builder/extensions/directus-extension-foo ./extensions/directus-extension-foo
COPY --from=builder /builder/extensions/directus-extension-bar ./extensions/directus-extension-bar

CMD ./newstart.sh

However it’s still spitting out a syntax error: unexpected word error when I try to deploy.

2023-08-21T06:28:16.132 app[9185e53f1ee948] lhr [info] INFO Starting init (commit: b437b5b)...
2023-08-21T06:28:16.154 app[9185e53f1ee948] lhr [info] INFO Mounting /dev/vdb at /directus w/ uid: 1000, gid: 1000 and chmod 0755
2023-08-21T06:28:16.159 app[9185e53f1ee948] lhr [info] INFO Resized /directus to 1069547520 bytes
2023-08-21T06:28:16.160 app[9185e53f1ee948] lhr [info] INFO Preparing to run: `sh /bin/sh -c ./newstart.sh` as node
2023-08-21T06:28:16.167 app[9185e53f1ee948] lhr [info] INFO [fly api proxy] listening at /.fly/api
2023-08-21T06:28:16.170 app[9185e53f1ee948] lhr [info] 2023/08/21 06:28:16 listening on [fdaa:2:5f23:a7b:2808:8ddf:75a9:2]:22 (DNS: [fdaa::3]:53)
2023-08-21T06:28:16.174 app[9185e53f1ee948] lhr [info] /bin/sh: line 3: syntax error: unexpected word (expecting ")")
2023-08-21T06:28:17.171 app[9185e53f1ee948] lhr [info] INFO Main child exited normally with code: 2
2023-08-21T06:28:17.171 app[9185e53f1ee948] lhr [info] INFO Starting clean up.
2023-08-21T06:28:17.172 app[9185e53f1ee948] lhr [info] INFO Umounting /dev/vdb from /directus
2023-08-21T06:28:17.173 app[9185e53f1ee948] lhr [info] ERROR error umounting /directus: EBUSY: Device or resource busy, retrying in a bit
2023-08-21T06:28:17.919 app[9185e53f1ee948] lhr [info] ERROR error umounting /directus: EBUSY: Device or resource busy, retrying in a bit
2023-08-21T06:28:18.670 app[9185e53f1ee948] lhr [info] ERROR error umounting /directus: EBUSY: Device or resource busy, retrying in a bit
2023-08-21T06:28:19.421 app[9185e53f1ee948] lhr [info] ERROR error umounting /directus: EBUSY: Device or resource busy, retrying in a bit
2023-08-21T06:28:20.177 app[9185e53f1ee948] lhr [info] WARN hallpass exited, pid: 264, status: signal: 15 (SIGTERM)
2023-08-21T06:28:20.179 app[9185e53f1ee948] lhr [info] 2023/08/21 06:28:20 listening on [fdaa:2:5f23:a7b:2808:8ddf:75a9:2]:22 (DNS: [fdaa::3]:53)
2023-08-21T06:28:21.174 app[9185e53f1ee948] lhr [info] [ 5.311183] reboot: Restarting system
2023-08-21T06:28:21.304 runner[9185e53f1ee948] lhr [info] machine did not have a restart policy, defaulting to restart
2023-08-21T06:28:21.687 app[9185e53f1ee948] lhr [info] [ 0.040754] PCI: Fatal: No config space access function found

rubys · August 21, 2023, 5:05pm

OK, this is definitely weird.

I’ve reproduced the symptoms (namely syntax error: unexpected word (expecting ")")) on my Linux development machine. I have NOT yet reproduced them on fly.io.

The problem is that in the above quoted command, there are too many shells. It appears to be trying to run bin/sh (which is a binary executable) as a shell script and reporting a parsing error.

If I had been able to reproduce this using a Dockerfile deployed to fly.io, I would feel more confident in the recommendation, but try changing CMD to ENTRYPOINT in your Dockerfile.

MattClegg · August 21, 2023, 5:52pm

Thanks for checking that. Changing the Dockerfile to use an ENTRYPOINT seemed to help:

ENTRYPOINT ./newstart.sh

I’ve also changed the fly.toml slightly:


app = "directus_demo"
primary_region = "lhr"
kill_signal = "SIGINT"
kill_timeout = 15

[env]
  KEY="08cfc17e-1175-4f55-b15e-a3e36c45eae5"
  SECRET="0e946813-36b0-487b-82f2-854164ddfbf8"
  DB_CLIENT="sqlite3"
  DB_FILENAME="/directus/database/data.db"
  STORAGE_LOCATIONS="local"
  STORAGE_LOCAL_DRIVER="local"
  STORAGE_LOCAL_ROOT="/directus/uploads"
  ADMIN_EMAIL="admin@example.com"
  ADMIN_PASSWORD="HelloWorld123"
  PUBLIC_URL="https://directus_demo.fly.dev"
  PORT=8055

[build]
  dockerfile = ".\\Dockerfile"

[mounts]
  source="demo_data"
  destination="/directus"

[[services]]
  internal_port = 8055
  processes = ["app"]
  protocol = "tcp"
  script_checks = []
  [services.concurrency]
    hard_limit = 25
    soft_limit = 20
    type = "connections"

  [[services.http_checks]]
    grace_period = "30s"
    interval = 10000
    method = "get"
    path = "/server/health"
    protocol = "http"
    timeout = 2000
    tls_skip_verify = false
    [services.http_checks.headers]

  [[services.ports]]
    force_https = true
    handlers = ["http"]
    port = 80

  [[services.ports]]
    handlers = ["tls", "http"]
    port = 443

  [[services.tcp_checks]]
    grace_period = "1s"
    interval = "15s"
    restart_limit = 0
    timeout = "2s"

However it still doesn’t want to boot:

 2023-08-21T17:48:01.640 runner[9185e53f1ee948] lhr [info] Pulling container image registry.fly.io/mcc-backend:deployment-01H8CKBP85KCW8DJ7QMDZ96A1S
2023-08-21T17:48:01.890 runner[9185e53f1ee948] lhr [info] Successfully prepared image registry.fly.io/mcc-backend:deployment-01H8CKBP85KCW8DJ7QMDZ96A1S (249.501337ms)
2023-08-21T17:48:01.910 runner[9185e53f1ee948] lhr [info] Setting up volume 'mcc_backend_data'
2023-08-21T17:48:01.910 runner[9185e53f1ee948] lhr [info] Opening encrypted volume
2023-08-21T17:48:02.299 runner[9185e53f1ee948] lhr [info] Configuring firecracker
2023-08-21T17:48:02.775 app[9185e53f1ee948] lhr [info] [ 0.042759] PCI: Fatal: No config space access function found
2023-08-21T17:48:03.008 app[9185e53f1ee948] lhr [info] INFO Starting init (commit: b437b5b)...
2023-08-21T17:48:03.030 app[9185e53f1ee948] lhr [info] INFO Mounting /dev/vdb at /directus w/ uid: 1000, gid: 1000 and chmod 0755
2023-08-21T17:48:03.033 app[9185e53f1ee948] lhr [info] INFO Resized /directus to 1069547520 bytes
2023-08-21T17:48:03.034 app[9185e53f1ee948] lhr [info] INFO Preparing to run: `sh` as node
2023-08-21T17:48:03.045 app[9185e53f1ee948] lhr [info] INFO [fly api proxy] listening at /.fly/api
2023-08-21T17:48:03.048 app[9185e53f1ee948] lhr [info] 2023/08/21 17:48:03 listening on [fdaa:2:5f23:a7b:2808:8ddf:75a9:2]:22 (DNS: [fdaa::3]:53)
2023-08-21T17:48:03.167 health[9185e53f1ee948] lhr [warn] Health check on port 8055 is in a 'warning' state. Your app may not be responding properly. Services exposed on ports [80, 443] may have intermittent failures until the health check passes.
2023-08-21T17:48:03.167 health[9185e53f1ee948] lhr [warn] Health check on port 8055 is in a 'warning' state. Your app may not be responding properly. Services exposed on ports [80, 443] may have intermittent failures until the health check passes.
2023-08-21T17:48:04.045 app[9185e53f1ee948] lhr [info] INFO Main child exited normally with code: 0
2023-08-21T17:48:04.046 app[9185e53f1ee948] lhr [info] INFO Starting clean up.
2023-08-21T17:48:04.046 app[9185e53f1ee948] lhr [info] INFO Umounting /dev/vdb from /directus
2023-08-21T17:48:04.047 app[9185e53f1ee948] lhr [info] ERROR error umounting /directus: EBUSY: Device or resource busy, retrying in a bit
2023-08-21T17:48:04.800 app[9185e53f1ee948] lhr [info] ERROR error umounting /directus: EBUSY: Device or resource busy, retrying in a bit
2023-08-21T17:48:05.553 app[9185e53f1ee948] lhr [info] ERROR error umounting /directus: EBUSY: Device or resource busy, retrying in a bit
2023-08-21T17:48:06.306 app[9185e53f1ee948] lhr [info] ERROR error umounting /directus: EBUSY: Device or resource busy, retrying in a bit
2023-08-21T17:48:07.064 app[9185e53f1ee948] lhr [info] WARN hallpass exited, pid: 264, status: signal: 15 (SIGTERM)
2023-08-21T17:48:07.069 app[9185e53f1ee948] lhr [info] 2023/08/21 17:48:07 listening on [fdaa:2:5f23:a7b:2808:8ddf:75a9:2]:22 (DNS: [fdaa::3]:53)
2023-08-21T17:48:08.065 app[9185e53f1ee948] lhr [info] [ 5.331257] reboot: Restarting system
2023-08-21T17:48:08.179 runner[9185e53f1ee948] lhr [info] machine exited with exit code 0, not restarting

Is it just me or does it seem to be exiting as soon as it starts?

rubys · August 21, 2023, 8:09pm

Weird things continue to happen here. Now you are running sh with no parameters. If the system shell is run with no parameters and no stdin, yes it will exit immediately.

Looking around for things that are odd, the following catches my eye:

Backslashes are what you would see on Windows, not on Linux. Since the default for this is Dockerfile in your top directory, perhaps these lines can be deleted entirely?

In any case, what you want is a combination of ENTRYPOINT and/or CMD that runs:

sh -c ./newstart.sh

system · August 28, 2023, 8:09pm

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.