Machine stuck in restart loop. Destroying does not solve problem

Questions / Help
1

I tried to deploy earlier today and I got an error
“swapon Invalid argument”
I was surprised because that line in my Dockerfile had been working previously without any issues however after spending hours searching I found that it’s recommended to specify swap_size_mb in the fly.toml file instead. I made the necessary changes but now the machine is stuck in a restart loop and I’m unable to make a successful deployment because of a timeout waiting for health checks to pass on the machine.

I have destroyed the machine several times but it does not solve the issue. This is a slice of the live logs.

Starting machine

2024-04-19T13:47:13.207 app[4d89d965b74638] sjc [info] [ 0.038875] Spectre V2 : WARNING: Unprivileged eBPF is enabled with eIBRS on, data leaks possible via Spectre v2 BHB attacks!

2024-04-19T13:47:13.234 app[4d89d965b74638] sjc [info] [ 0.042280] PCI: Fatal: No config space access function found

2024-04-19T13:47:13.466 app[4d89d965b74638] sjc [info] INFO Starting init (commit: 65db7f7)…

2024-04-19T13:47:13.482 app[4d89d965b74638] sjc [info] INFO Preparing to run: docker-entrypoint.sh node as root

2024-04-19T13:47:13.490 app[4d89d965b74638] sjc [info] INFO [fly api proxy] listening at /.fly/api

2024-04-19T13:47:13.498 app[4d89d965b74638] sjc [info] 2024/04/19 13:47:13 listening on [fdaa:0:ce68:a7b:247:28ce:25f1:2]:22 (DNS: [fdaa::3]:53)

2024-04-19T13:47:13.538 runner[4d89d965b74638] sjc [info] Machine started in 489ms

2024-04-19T13:47:14.491 app[4d89d965b74638] sjc [info] INFO Main child exited normally with code: 0

2024-04-19T13:47:14.504 app[4d89d965b74638] sjc [info] INFO Starting clean up.

2024-04-19T13:47:14.510 app[4d89d965b74638] sjc [info] WARN could not unmount /rootfs: EINVAL: Invalid argument

2024-04-19T13:47:14.510 app[4d89d965b74638] sjc [info] [ 1.315311] reboot: Restarting system

2024-04-19T13:47:14.543 proxy[4d89d965b74638] sjc [info] machine started in 1.505011459s

2024-04-19T13:47:15.385 runner[4d89d965b74638] sjc [info] machine exited with exit code 0, not restarting

2024-04-19T13:47:18.379 proxy[4d89d965b74638] sjc [info] waiting for machine to be reachable on 0.0.0.0:8080 (waited 18.372815283s so far)

2024-04-19T13:47:19.342 proxy[4d89d965b74638] sjc [info] Starting machine

2024-04-19T13:47:19.508 app[4d89d965b74638] sjc [info] [ 0.037548] Spectre V2 : WARNING: Unprivileged eBPF is enabled with eIBRS on, data leaks possible via Spectre v2 BHB attacks!

2024-04-19T13:47:19.534 app[4d89d965b74638] sjc [info] [ 0.041026] PCI: Fatal: No config space access function found

2024-04-19T13:47:19.762 app[4d89d965b74638] sjc [info] INFO Starting init (commit: 65db7f7)…

2024-04-19T13:47:19.777 app[4d89d965b74638] sjc [info] INFO Preparing to run: docker-entrypoint.sh node as root

2024-04-19T13:47:19.785 app[4d89d965b74638] sjc [info] INFO [fly api proxy] listening at /.fly/api

2024-04-19T13:47:19.793 app[4d89d965b74638] sjc [info] 2024/04/19 13:47:19 listening on [fdaa:0:ce68:a7b:247:28ce:25f1:2]:22 (DNS: [fdaa::3]:53)

2024-04-19T13:47:19.801 runner[4d89d965b74638] sjc [info] Machine started in 442ms

2024-04-19T13:47:20.544 proxy[4d89d965b74638] sjc [info] waiting for machine to be reachable on 0.0.0.0:8080 (waited 6.000563106s so far)

2024-04-19T13:47:20.786 app[4d89d965b74638] sjc [info] INFO Main child exited normally with code: 0

2024-04-19T13:47:20.800 app[4d89d965b74638] sjc [info] INFO Starting clean up.

2024-04-19T13:47:20.805 app[4d89d965b74638] sjc [info] WARN could not unmount /rootfs: EINVAL: Invalid argument

2024-04-19T13:47:20.805 proxy[4d89d965b74638] sjc [info] machine started in 1.463658031s

2024-04-19T13:47:20.806 app[4d89d965b74638] sjc [info] [ 1.309528] reboot: Restarting system

2024-04-19T13:47:21.692 runner[4d89d965b74638] sjc [info] machine exited with exit code 0, not restarting

2024-04-19T13:47:24.383 proxy[4d89d965b74638] sjc [info] waiting for machine to be reachable on 0.0.0.0:8080 (waited 24.377103836s so far)

2024-04-19T13:47:25.850 proxy[4d89d965b74638] sjc [info] waiting for machine to be reachable on 0.0.0.0:8080 (waited 11.307009575s so far)

2024-04-19T13:47:26.889 proxy[4d89d965b74638] sjc [info] waiting for machine to be reachable on 0.0.0.0:8080 (waited 6.083688277s so far)

2

I mistakenly commented out the line that starts my app while removing the command that creates swap memory in the Dockerfile so the machine just shuts down because nothing happens after it boots. Everything works fine now.

3

If you do need to use swap, the supported way is in the fly.toml. See A new way to RootFS - #2 by JP_Phillips for details on how and why.

1 Like
4

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.