Pre-built image works locally, but on fly: `sh: line 1: gunicorn: command not found`

1

I have an image which I am building with Nix, which runs ok on my machine
The nix expression:

packages.docker = pkgs.dockerTools.streamLayeredImage {
	name = "nickel_club";
	contents = [ 
		pkgs.nickel_club
		pkgs.nickel_club.dependencyEnv

		# TODO remove debugging deps
		pkgs.coreutils 
		pkgs.iputils
		pkgs.inetutils
		pkgs.bash
		pkgs.which
	];
	config = {
		Cmd = [ "sh" "-c" "gunicorn --bind 0.0.0.0:\${PORT} 'nickel_club:create_app()'"]; 
	};
};

Running it works:

⮞ docker run -itp 8080:8080 --env-file .env registry.fly.io/nickel-club:latest
[2022-09-13 06:47:51 +0000] [1] [INFO] Starting gunicorn 20.1.0
[2022-09-13 06:47:51 +0000] [1] [INFO] Listening at: http://0.0.0.0:8080 (1)
< ... snip >

I push to the fly registry successfully

⮞ docker push registry.fly.io/nickel-club:latest
< ... snip >
latest: digest: sha256:7e3757875902c9b032697de3d489ce4b2f8a70db0a973cc8b07a0d1f44a37014 size: 12276

Now I do a deploy with this fly.toml

# fly.toml file generated for nickel-club on 2022-09-11T13:06:12+10:00

app = "nickel-club"
kill_signal = "SIGINT"
kill_timeout = 5
processes = []

[build]
  image = "registry.fly.io/nickel-club:latest"

[env]

[experimental]
  allowed_public_ports = []
  auto_rollback = true

[[services]]
  http_checks = []
  internal_port = 8080
  processes = ["app"]
  protocol = "tcp"
  script_checks = []
  [services.concurrency]
    hard_limit = 25
    soft_limit = 20
    type = "connections"

  [[services.ports]]
    force_https = true
    handlers = ["http"]
    port = 80

  [[services.ports]]
    handlers = ["tls", "http"]
    port = 443

  [[services.tcp_checks]]
    grace_period = "1s"
    interval = "15s"
    restart_limit = 0
    timeout = "2s"

and get an error sh: line 1: gunicorn: command not found:

⮞ flyctl deploy
Update available 0.0.372 -> v0.0.390.
Run "flyctl version update" to upgrade.
==> Verifying app config
--> Verified app config
==> Building image
Searching for image 'registry.fly.io/nickel-club:latest' remotely...
image found: img_ox20prj99d8vj1zq
==> Creating release
--> release v4 created

--> You can detach the terminal anytime without stopping the deployment
==> Monitoring deployment

 1 desired, 1 placed, 0 healthy, 1 unhealthy [restarts: 2]
Failed Instances

Failure #1

Instance
ID      	PROCESS	VERSION	REGION	DESIRED	STATUS	HEALTH CHECKS	RESTARTS	CREATED
e561253a	       	4      	syd   	stop   	failed	             	2       	1m7s   	

Recent Events
TIMESTAMP           	TYPE           	MESSAGE
2022-09-13T06:56:57Z	Received       	Task received by client
2022-09-13T06:56:57Z	Task Setup     	Building Task Directory
2022-09-13T06:57:05Z	Started        	Task started by client
2022-09-13T06:57:07Z	Terminated     	Exit Code: 127
2022-09-13T06:57:07Z	Restarting     	Task restarting in 1.038070838s
2022-09-13T06:57:14Z	Started        	Task started by client
2022-09-13T06:57:16Z	Terminated     	Exit Code: 127
2022-09-13T06:57:16Z	Restarting     	Task restarting in 1.057473798s
2022-09-13T06:57:23Z	Started        	Task started by client
2022-09-13T06:57:25Z	Terminated     	Exit Code: 127
2022-09-13T06:57:25Z	Not Restarting 	Exceeded allowed attempts 2 in interval 5m0s and mode is "fail"
2022-09-13T06:57:25Z	Alloc Unhealthy	Unhealthy because of failed task
2022-09-13T06:57:26Z	Killing        	Sent interrupt. Waiting 5s before force killing

2022-09-13T06:57:05Z   [info]Configuring firecracker
2022-09-13T06:57:05Z   [info]Starting virtual machine
2022-09-13T06:57:05Z   [info]Starting init (commit: 249766e)...
2022-09-13T06:57:05Z   [info]Preparing to run: `sh -c gunicorn --bind 0.0.0.0:${PORT} 'nickel_club:create_app()'` as root
2022-09-13T06:57:05Z   [info]2022/09/13 06:57:05 listening on [fdaa:0:9d04:a7b:8566:e561:253a:2]:22 (DNS: [fdaa::3]:53)
2022-09-13T06:57:06Z   [info]Starting clean up.
2022-09-13T06:57:12Z   [info]Starting instance
2022-09-13T06:57:13Z   [info]Configuring virtual machine
2022-09-13T06:57:13Z   [info]Pulling container image
2022-09-13T06:57:14Z   [info]Unpacking image
2022-09-13T06:57:14Z   [info]Preparing kernel init
2022-09-13T06:57:14Z   [info]Configuring firecracker
2022-09-13T06:57:14Z   [info]Starting virtual machine
2022-09-13T06:57:14Z   [info]Starting init (commit: 249766e)...
2022-09-13T06:57:14Z   [info]Preparing to run: `sh -c gunicorn --bind 0.0.0.0:${PORT} 'nickel_club:create_app()'` as root
2022-09-13T06:57:14Z   [info]2022/09/13 06:57:14 listening on [fdaa:0:9d04:a7b:8566:e561:253a:2]:22 (DNS: [fdaa::3]:53)
2022-09-13T06:57:14Z   [info]sh: line 1: gunicorn: command not found
2022-09-13T06:57:15Z   [info]Starting clean up.
2022-09-13T06:57:21Z   [info]Starting instance
2022-09-13T06:57:22Z   [info]Configuring virtual machine
2022-09-13T06:57:22Z   [info]Pulling container image
2022-09-13T06:57:23Z   [info]Unpacking image
2022-09-13T06:57:23Z   [info]Preparing kernel init
2022-09-13T06:57:23Z   [info]Configuring firecracker
2022-09-13T06:57:23Z   [info]Starting virtual machine
2022-09-13T06:57:23Z   [info]Starting init (commit: 249766e)...
2022-09-13T06:57:23Z   [info]Preparing to run: `sh -c gunicorn --bind 0.0.0.0:${PORT} 'nickel_club:create_app()'` as root
2022-09-13T06:57:23Z   [info]2022/09/13 06:57:23 listening on [fdaa:0:9d04:a7b:8566:e561:253a:2]:22 (DNS: [fdaa::3]:53)
2022-09-13T06:57:23Z   [info]sh: line 1: gunicorn: command not found
2022-09-13T06:57:24Z   [info]Starting clean up.
--> v4 failed - Failed due to unhealthy allocations - rolling back to job version 3 and deploying as v5

--> Troubleshooting guide at https://fly.io/docs/getting-started/troubleshooting/
Error abort

What am I doing wrong? Why is there a difference in behaviour between my local machine and the deploy environment?

Please let me know if there are any other details I should post.

1 Like