Newbie question. My app hosted on fly is down. My main app and the builder show a “pause” icon on the dashboard (see below).
How do i restart them?
How do i roll back the last functioning release in case that’s the issue?
I think the problem started with a failed deployment, shouldn’t fly.io automatically roll back to previous release?
Hi @titocosta
Without knowing much about your app, I can point you to a few things to check into.
The pause icon alone doesn’t mean your app is down, it could just mean that your app’s Machines are in a stopped state.
Depending on your configuration, this might be because of the auto start / stop feature. If that’s the case, then your app’s Machines will start up again when there’s a connection/request to your app.
If your app is unreachable, then there may be another issue. You can start by making sure you have the latest version of flyctl (run fly version update). And then try fly deploy again.
If deployment fails, try checking the logs during deployment and post back here.
My site openindex.ai is down and not reachable. Both the builder and openindex app are suspended.
I am not using any auto start and stop feature, so something else suspended those apps, but I don’t know what (i have not received any error message or notification).
I can’t deploy any more any version, I am stuck.
I tried to upgrade version, restart apps and machines and to redeploy.
I am stuck with site down.
Could you post your fly.toml file?
What do you see when you run fly status?
It would also be helpful to see the output you’re getting when you fly deploy.
My deploy fails with the error below. How do I roll back the previous release?
> next start
ready - started server on 0.0.0.0:8080, url: http://localhost:8080
info - Loaded env from /app/.env
npm ERR! code EACCES
npm ERR! syscall mkdir
npm ERR! path /nonexistent
npm ERR! errno -13
npm ERR! Error: EACCES: permission denied, mkdir '/nonexistent'
npm ERR! [Error: EACCES: permission denied, mkdir '/nonexistent'] {
npm ERR! errno: -13,
npm ERR! code: 'EACCES',
npm ERR! syscall: 'mkdir',
npm ERR! path: '/nonexistent'
npm ERR! }
npm ERR!
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR!
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR! permissions of the file and its containing directories, or try running
npm ERR! the command again as root/Administrator.
npm ERR! Log files were not written due to an error writing to the directory: /nonexistent/.npm/_logs
npm ERR! You can rerun the command with `--loglevel=verbose` to see the logs in your terminal
INFO Main child exited normally with code: 243
INFO Starting clean up.
WARN hallpass exited, pid: 515, status: signal: 15 (SIGTERM)
2023/06/15 18:24:01 listening on [fdaa:1:d3b0:a7b:a7:c21c:5328:2]:22 (DNS: [fdaa::3]:53)
[ 3.154304] reboot: Restarting system
INFO Starting init (commit: 0b28cec)...
Error: smoke checks for 3d8d9679b2d538 failed: the app appears to be crashing
fly.toml
fly status
I checked out the previous commit release from git to deploy it, but I am still getting the same error now:
npm ERR! syscall mkdir
npm ERR! path /nonexistent
Looks like some other things that has to do with npm installing packages. I don’t know what this “/nonexistent” URL refers to.
Something must have changed in the production environment that prevents it from deploying successfully.
Seems a directory access error when installing the packages with npm.
For now I managed to roll back to a previous release as described here while I figure out the deployment errors.
The only google result for error npm ERR! path /nonexistent is about this homebridge ui plugin Reddit - Dive into anything
My deploy process takes a very long time (6mins and counting on first step) with no feedback.
Not sure what changed or what is creating the issue.
It seems that the deploy eventually succeeds, but then checks fail.
➜ openindexnext git:(main) flyctl deploy -a openindex
==> Verifying app config
Validating /Users/titocosta/Developer/openindex/openindexnext/fly.toml
Platform: machines
✓ Configuration is valid
--> Verified app config
==> Building image
Remote builder fly-builder-holy-lake-7626 ready
==> Creating build context
--> Creating build context done
==> Building image with Docker
--> docker host: 20.10.12 linux x86_64
[+] Building 1313.1s (0/1)
[+] Building 1.9s (12/12) FINISHED
=> [internal] load remote build context 0.0s
=> copy /context / 0.4s
=> [internal] load metadata for docker.io/library/node:18 0.9s
=> [builder 1/5] FROM docker.io/library/node:18@sha256:4a55308cc855cba1a925d19ae4e45838741dad2fd7bb8949a93b2a0f2ae339e3 0.0s
=> CACHED [builder 2/5] WORKDIR /app 0.0s
=> CACHED [runner 3/5] RUN addgroup --system --gid 1001 nodejs 0.0s
=> CACHED [runner 4/5] RUN adduser --system --uid 1001 nextjs 0.0s
=> CACHED [builder 3/5] COPY . . 0.0s
=> CACHED [builder 4/5] RUN npm ci 0.0s
=> CACHED [builder 5/5] RUN npm run build 0.0s
=> CACHED [runner 5/5] COPY --from=builder /app ./ 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:063f3311937f5da25ff5bdcef30f38cd68bb53ac5ea90839430356f563569c34 0.0s
=> => naming to registry.fly.io/openindex:deployment-01H31W37PKAT5DP06S5321ZYTY 0.0s
--> Building image done
==> Pushing image to fly
The push refers to repository [registry.fly.io/openindex]
b267b8eaf79a: Layer already exists
4db96335c266: Layer already exists
6406f4ff0a61: Layer already exists
cbc4b9326cdc: Layer already exists
a57db338c0ef: Layer already exists
f273e3620919: Layer already exists
69208400c34b: Layer already exists
1fdf7be6e30b: Layer already exists
e67fb4bad8f4: Layer already exists
964529c819bb: Layer already exists
2f98f42985b1: Layer already exists
332b199f36eb: Layer already exists
deployment-01H31W37PKAT5DP06S5321ZYTY: digest: sha256:8afec26ac7a6d2d47cddee4ac847aeeed25ba1e04136ebdfe94681b4cb6f9bc6 size: 2839
--> Pushing image done
image: registry.fly.io/openindex:deployment-01H31W37PKAT5DP06S5321ZYTY
image size: 2.5 GB
Watch your app at https://fly.io/apps/openindex/monitoring
Updating existing machines in 'openindex' with rolling strategy
Machine 3d8d9679b2d538 [app] has state: started
[1/1] Checking that 3d8d9679b2d538 [app] is up and running
Smoke checks for 3d8d9679b2d538 failed: the app appears to be crashing
Check its logs: here's the last lines below, or run 'fly logs -i 3d8d9679b2d538':
Successfully prepared image registry.fly.io/openindex:deployment-01H31W37PKAT5DP06S5321ZYTY (1.54570387s)
Configuring firecracker
INFO Sending signal SIGINT to main child process w/ PID 514
INFO Sending signal SIGTERM to main child process w/ PID 514
INFO Main child exited with signal (with signal 'SIGTERM', core dumped? false)
INFO Starting clean up.
WARN hallpass exited, pid: 515, status: signal: 15 (SIGTERM)
2023/06/16 10:52:56 listening on [fdaa:1:d3b0:a7b:a7:c21c:5328:2]:22 (DNS: [fdaa::3]:53)
[53902.231086] reboot: Restarting system
INFO Starting init (commit: 0b28cec)...
INFO Preparing to run: `docker-entrypoint.sh npm run start` as nextjs
INFO [fly api proxy] listening at /.fly/api
2023/06/16 10:52:58 listening on [fdaa:1:d3b0:a7b:a7:c21c:5328:2]:22 (DNS: [fdaa::3]:53)
> openindexnext@0.2.78 start
> next start
ready - started server on 0.0.0.0:8080, url: http://localhost:8080
info - Loaded env from /app/.env
npm ERR! code EACCES
npm ERR! syscall mkdir
npm ERR! path /nonexistent
npm ERR! errno -13
npm ERR! Error: EACCES: permission denied, mkdir '/nonexistent'
npm ERR! [Error: EACCES: permission denied, mkdir '/nonexistent'] {
npm ERR! errno: -13,
npm ERR! code: 'EACCES',
npm ERR! syscall: 'mkdir',
npm ERR! path: '/nonexistent'
npm ERR! }
npm ERR!
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR!
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR! permissions of the file and its containing directories, or try running
npm ERR! the command again as root/Administrator.
npm ERR! Log files were not written due to an error writing to the directory: /nonexistent/.npm/_logs
npm ERR! You can rerun the command with `--loglevel=verbose` to see the logs in your terminal
INFO Main child exited normally with code: 243
INFO Starting clean up.
WARN hallpass exited, pid: 515, status: signal: 15 (SIGTERM)
2023/06/16 10:53:00 listening on [fdaa:1:d3b0:a7b:a7:c21c:5328:2]:22 (DNS: [fdaa::3]:53)
[ 3.158934] reboot: Restarting system
machine did not have a restart policy, defaulting to restart
INFO Starting init (commit: 0b28cec)...
INFO Preparing to run: `docker-entrypoint.sh npm run start` as nextjs
INFO [fly api proxy] listening at /.fly/api
2023/06/16 10:53:02 listening on [fdaa:1:d3b0:a7b:a7:c21c:5328:2]:22 (DNS: [fdaa::3]:53)
> openindexnext@0.2.78 start
> next start
ready - started server on 0.0.0.0:8080, url: http://localhost:8080
info - Loaded env from /app/.env
npm ERR! code EACCES
npm ERR! syscall mkdir
npm ERR! path /nonexistent
npm ERR! errno -13
npm ERR! Error: EACCES: permission denied, mkdir '/nonexistent'
npm ERR! [Error: EACCES: permission denied, mkdir '/nonexistent'] {
npm ERR! errno: -13,
npm ERR! code: 'EACCES',
npm ERR! syscall: 'mkdir',
npm ERR! path: '/nonexistent'
npm ERR! }
npm ERR!
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR!
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR! permissions of the file and its containing directories, or try running
npm ERR! the command again as root/Administrator.
npm ERR! Log files were not written due to an error writing to the directory: /nonexistent/.npm/_logs
npm ERR! You can rerun the command with `--loglevel=verbose` to see the logs in your terminal
INFO Main child exited normally with code: 243
INFO Starting clean up.
WARN hallpass exited, pid: 515, status: signal: 15 (SIGTERM)
2023/06/16 10:53:04 listening on [fdaa:1:d3b0:a7b:a7:c21c:5328:2]:22 (DNS: [fdaa::3]:53)
Error: smoke checks for 3d8d9679b2d538 failed: the app appears to be crashing
I reverted to previous release and the site is up, but the problem is still there and I can’t deploy anymore.
If you reverted to the previous version of your Docker image and your app is working, it might be a good idea to check what’s changed in the image?
The issue does seem to be a permissions issue and your app isn’t allowed to create any directories.
Here are some resources about similar issues that might help you troubleshoot:
Still haven’t found a solution.
.next is included in .dockerignore
chown argument to COPY --chown=nextjs:nodejs --from=builder /app ./ in the DockerfileRUN mkdir -p /nonexistent and RUN mkdir -p /app/nonexistent in the DockerfileIt looks like adding both lines
RUN mkdir -p /nonexistent
RUN chown nextjs:nodejs /nonexistent
to the Dockerfile before line COPY --chown=nextjs:nodejs --from=builder /app ./ fixed it.
I am not sure what package may require to write to /nonexistent maybe for temporary files.
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.