Node app deploy stalling at 'vX is being deployed'

Questions / Help Build debugging
1

I’m having real trouble deploying my node application to fly. It appears to progress until it gets to ‘v is being deployed’ and then it stalls.

fly deploy -c fly.dev.toml 
==> Verifying app config
--> Verified app config
==> Building image
Remote builder fly-builder-wandering-forest-6038 ready
==> Creating build context
--> Creating build context done
==> Building image with Docker
--> docker host: 20.10.12 linux x86_64
[+] Building 0.5s (0/1)                                                                                                                                                                                             
[+] Building 0.6s (10/10) FINISHED                                                                                                                                                                                  
 => [internal] load remote build context                                                                                                                                                                       0.0s
 => copy /context /                                                                                                                                                                                            0.1s
 => [internal] load metadata for docker.io/library/node:16                                                                                                                                                     0.4s
 => [1/6] FROM docker.io/library/node:16@sha256:367b3b89399e6bd52746180c7b348c313015fc637d06a1f0e0ccb983fd52bfd1                                                                                               0.0s
 => CACHED [2/6] WORKDIR /app                                                                                                                                                                                  0.0s
 => CACHED [3/6] COPY package*.json ./                                                                                                                                                                         0.0s
 => CACHED [4/6] RUN apt-get update && apt-get install -y sqlite3                                                                                                                                              0.0s
 => CACHED [5/6] RUN npm install                                                                                                                                                                               0.0s
 => [6/6] COPY . .                                                                                                                                                                                             0.0s
 => exporting to image                                                                                                                                                                                         0.0s
 => => exporting layers                                                                                                                                                                                        0.0s
 => => writing image sha256:2c55198348bd4b03c618b043872cfbd30bf32d3575d039ebd1bc92d3728ddb3c                                                                                                                   0.0s
 => => naming to registry.fly.io/u17cc-dev:deployment-01GRRYMGT2NS0CWB1RDZZMY9MQ                                                                                                                               0.0s
--> Building image done
==> Pushing image to fly
The push refers to repository [registry.fly.io/u17cc-dev]
3b11d282e263: Pushed 
27f91441ce56: Layer already exists 
c4ce33872602: Layer already exists 
b81ad6b84e8f: Layer already exists 
928e444cce36: Layer already exists 
024c0b0eec04: Layer already exists 
3b7c989449c5: Layer already exists 
e2c36140d36a: Layer already exists 
4197f7214e01: Layer already exists 
d8fa520091e2: Layer already exists 
ea91e8fcfcc8: Layer already exists 
567d43b0f38b: Layer already exists 
a205a7bbdfe0: Layer already exists 
768661b0a155: Layer already exists 
deployment-01GRRYMGT2NS0CWB1RDZZMY9MQ: digest: sha256:78492af10164d1a6024143e55669984363340cf8828a62776c0279d197b6841f size: 3267
--> Pushing image done
image: registry.fly.io/u17cc-dev:deployment-01GRRYMGT2NS0CWB1RDZZMY9MQ
image size: 1.2 GB
==> Creating release
--> release v73 created

--> You can detach the terminal anytime without stopping the deployment
==> Monitoring deployment
Logs: https://fly.io/apps/u17cc-dev/monitoring

v73 is being deployed

The toml file is

# fly.toml file generated for u17cc on 2022-05-29T10:59:36+01:00

app = "u17cc-dev"

kill_signal = "SIGINT"
kill_timeout = 5
processes = []

[env]
  PORT = "8080"

[mounts]
source="data_dev"
destination="/data"

[experimental]
  allowed_public_ports = []
  auto_rollback = true

[[services]]
  http_checks = []
  internal_port = 8080
  processes = ["app"]
  protocol = "tcp"
  script_checks = []

  [services.concurrency]
    hard_limit = 25
    soft_limit = 20
    type = "connections"

  [[services.ports]]
    force_https = true
    handlers = ["http"]
    port = 80

  [[services.ports]]
    handlers = ["tls", "http"]
    port = 443

  [[services.tcp_checks]]
    grace_period = "1s"
    interval = "15s"
    restart_limit = 0
    timeout = "2s"

I’ve previously had great success deploying and running the app, but at some point in the past few weeks it has been extremely hit and miss.

2

This is most often an unintentional canary deploy. If you’re using volumes, or have scale count X --max-per-region=1, the orchestrator can’t start an additional vm. It needs an existing one to go away first.

If this is the case, you can get your deployment unstuck by stopping an old vm. Run fly status, then fly vm stop <id> and it should get moving.

You can avoid this in the future by running fly deploy --strategy rolling.