Machine fails if a volume is attached

Hi Fly Community,

I observe a strange issue that I don’t know how to resolve.

I am testing wundergraph/wunderbase, a SQLite/Prisma/Go stack that is supposed to scale to zero when no requests come in for 10 seconds. This works fine when no volume exists.

However, when I create and attach a volume in the process of deploying the app, the machine starts but does not serve any requests. And it never goes into suspended state.

Any idea why attaching a volume makes the machine fail to run correctly?

These are the steps I do for launching the machine, building the app, and running the machine with the volume attached:

 ~/d/w/testing-wunderbase> fly machine launch
? Choose an app name (leave blank to generate one): testing-wunderbase
automatically selected personal organization: Christoph Berger
Created app testing-wunderbase in org personal
? Select regions: Frankfurt, Germany (fra)
? Does this app run an HTTP service? Yes
? Which port does your service listen on? 4466
Wrote to fly.toml
Your app is ready. Deploy with `flyctl deploy`

## At this point, I added 'image = "wundergraph/wunderbase:latest"' to the fly.toml

~/d/w/testing-wunderbase> fly deploy --build-only
==> Verifying app config
--> Verified app config
==> Building image
Searching for image 'wundergraph/wunderbase' remotely...
image found: img_98dgp8nr0elpxw05

 ~/d/w/testing-wunderbase> fly volumes create testing_wunderbase
? Select regions: Frankfurt, Germany (fra)
        ID: vol_2n0l9vlj66m4635d
      Name: testing_wunderbase
       App: testing-wunderbase
    Region: fra
      Zone: e79a
   Size GB: 3
 Encrypted: true
Created at: 30 Sep 22 06:19 UTC

 ~/d/w/testing-wunderbase> fly machine run wundergraph/wunderbase -v testing_wunderbase:/app/data -c fly.toml
Searching for image 'wundergraph/wunderbase' remotely...
image found: img_98dgp8nr0elpxw05
Image: registry-1.docker.io/wundergraph/wunderbase:latest
Image size: 38 MB

Success! A machine has been successfully launched, waiting for it to be started
 Machine ID: e784370ce74783
 Instance ID: 01GE6GSZ63CSAHGY896D396FTW
 State: created
Machine started, you can connect via the following private ip
  fdba:0:3c57:a7c:23c0:b2c2:c4b3:2

 ~/d/w/testing-wunderbase> fly logs
2022-09-30T06:19:42Z runner[e784370ce74783] fra [info]Reserved resources for machine 'e784370ce74783'
2022-09-30T06:19:42Z runner[e784370ce74783] fra [info]Pulling container image
2022-09-30T06:19:42Z runner[e784370ce74783] fra [info]Pulling image failed
2022-09-30T06:19:42Z runner[e784370ce74783] fra [info]Pulling container image
2022-09-30T06:19:43Z runner[e784370ce74783] fra [info]Unpacking image
2022-09-30T06:19:49Z runner[e784370ce74783] fra [info]Setting up volume 'testing_wunderbase'
2022-09-30T06:19:49Z runner[e784370ce74783] fra [info]Uninitialized volume 'testing_wunderbase', initializing...
2022-09-30T06:19:49Z runner[e784370ce74783] fra [info]Encrypting volume
2022-09-30T06:19:56Z runner[e784370ce74783] fra [info]Opening encrypted volume
2022-09-30T06:19:58Z runner[e784370ce74783] fra [info]Formatting volume
2022-09-30T06:19:58Z runner[e784370ce74783] fra [info]Configuring firecracker
2022-09-30T06:19:59Z app[e784370ce74783] fra [info]2022/09/30 06:19:59 Migration successful, updating lock file
2022-09-30T06:19:59Z app[e784370ce74783] fra [info]2022/09/30 06:19:59 Server Listening on: http://0.0.0.0:4466
2022-09-30T06:19:59Z app[e784370ce74783] fra [info]Error: exec: "bash": executable file not found in $PATH
^C

## The bash error is benign. It also occurs when no volume exists and the machine runs fine.

## The machine has the volume attached. It is in "started" status. With no volume, the machine goes into "stopped" mode after 10 seconds of inactivity.

127 ~/d/w/testing-wunderbase> fly machine list
1 machines have been retrieved.
View them in the UI here

testing-wunderbase
ID            	NAME             	STATE  	REGION	IMAGE                        	IP ADDRESS                      	VOLUME            	CREATED             	LAST UPDATED
e784370ce74783	dawn-feather-6564	started	fra   	wundergraph/wunderbase:latest	fdab:0:3b57:a7b:23c6:b2c2:c4b3:2	testing_wunderbase	2022-09-30T06:19:42Z	2022-09-30T06:19:59Z

Edited to add:

Without a volume, the machine works fine, except that all updates are lost when the machine suspends. (Expected behavior.)

 ~/d/w/testing-wunderbase> fly machine launch
? Choose an app name (leave blank to generate one): testing-wunderbase
automatically selected personal organization: Christoph Berger
Created app testing-wunderbase in org personal
? Select regions: Frankfurt, Germany (fra)
? Does this app run an HTTP service? Yes
? Which port does your service listen on? 4466
Wrote to fly.toml
Your app is ready. Deploy with `flyctl deploy`

## Add 'image = "wundergraph/wunderbase:latest"' to fly.toml

 ~/d/w/testing-wunderbase> fly deploy
==> Verifying app config
--> Verified app config
==> Building image
Searching for image 'wundergraph/wunderbase:latest' remotely...
image found: img_98dgp8nr0elpxw05
Deploying with rolling strategy 🌎 Launching VM with image registry-1.docker.io/wundergraph/wunderbase:latest
Deploying with rolling strategy ✓
 ~/d/w/testing-wunderbase>

2022-09-30T06:45:07Z runner[e784901be23283] fra [info]Reserved resources for machine 'e784901be23283'
2022-09-30T06:45:07Z runner[e784901be23283] fra [info]Pulling container image
2022-09-30T06:45:08Z runner[e784901be23283] fra [info]Unpacking image
2022-09-30T06:45:09Z runner[e784901be23283] fra [info]Configuring firecracker
2022-09-30T06:45:09Z app[e784901be23283] fra [info]2022/09/30 06:45:09 Migration successful, updating lock file
2022-09-30T06:45:09Z app[e784901be23283] fra [info]2022/09/30 06:45:09 Server Listening on: http://0.0.0.0:4466
2022-09-30T06:45:09Z app[e784901be23283] fra [info]Error: exec: "bash": executable file not found in $PATH
2022-09-30T06:45:50Z proxy[e784901be23283] fra [info]Machine became reachable in 10.654915ms
2022-09-30T06:46:04Z app[e784901be23283] fra [info]No requests for 10 seconds, cancelling context
2022-09-30T06:46:04Z app[e784901be23283] fra [info]2022/09/30 06:46:04 Server stopped
2022-09-30T06:46:04Z app[e784901be23283] fra [info]2022/09/30 06:46:04 Query Engine stopped
2022-09-30T06:46:07Z runner[e784901be23283] fra [info]machine exited with exit code 0, not restarting
2022-09-30T06:46:23Z app[e784901be23283] fra [info]2022/09/30 06:46:23 Migration successful, updating lock file
2022-09-30T06:46:23Z app[e784901be23283] fra [info]2022/09/30 06:46:23 Server Listening on: http://0.0.0.0:4466
2022-09-30T06:46:23Z app[e784901be23283] fra [info]Error: exec: "bash": executable file not found in $PATH
2022-09-30T06:46:23Z proxy[e784901be23283] fra [info]Machine started in 318.127939ms, waiting for it to listen on 0.0.0.0:4466
2022-09-30T06:46:23Z proxy[e784901be23283] fra [info]Machine became reachable in 13.110009ms
2022-09-30T06:47:18Z app[e784901be23283] fra [info]2022/09/30 06:47:18 Server stopped
2022-09-30T06:47:18Z app[e784901be23283] fra [info]2022/09/30 06:47:18 Query Engine stopped
2022-09-30T06:47:18Z app[e784901be23283] fra [info]No requests for 10 seconds, cancelling context
2022-09-30T06:47:21Z runner[e784901be23283] fra [info]machine exited with exit code 0, not restarting
2022-09-30T06:47:22Z app[e784901be23283] fra [info]2022/09/30 06:47:22 Migration successful, updating lock file
2022-09-30T06:47:22Z app[e784901be23283] fra [info]2022/09/30 06:47:22 Server Listening on: http://0.0.0.0:4466
2022-09-30T06:47:22Z app[e784901be23283] fra [info]Error: exec: "bash": executable file not found in $PATH
2022-09-30T06:47:22Z proxy[e784901be23283] fra [info]Machine started in 332.490538ms, waiting for it to listen on 0.0.0.0:4466
2022-09-30T06:47:22Z proxy[e784901be23283] fra [info]Machine became reachable in 11.256002ms
2022-09-30T06:47:37Z app[e784901be23283] fra [info]2022/09/30 06:47:37 Server stopped
2022-09-30T06:47:37Z app[e784901be23283] fra [info]2022/09/30 06:47:37 Query Engine stopped
2022-09-30T06:47:37Z app[e784901be23283] fra [info]No requests for 10 seconds, cancelling context
2022-09-30T06:47:39Z runner[e784901be23283] fra [info]machine exited with exit code 0, not restarting

Hey @christoph1 ,

What version of flyctl are you running currently?

Hi @shaun,

fly v0.0.399 darwin/amd64 Commit: 3f98b0dd BuildDate: 2022-09-29T21:50:22Z