Rails Failing Health Check Port 3000

Questions / Help
1

I’m using the following fly.toml config:

app = 'name-of-the-app'
primary_region = 'cdg'
console_command = '/app/bin/rails console'

[build]
  build-target = 'production'

[deploy]
  release_command = './bin/rails db:prepare'

[processes]
  app = './bin/rails server'
  sidekiq = 'bundle exec sidekiq'

[http_service]
  internal_port = 3000
  force_https = true
  auto_stop_machines = 'stop'
  auto_start_machines = true
  min_machines_running = 0
  processes = ['app']

[[http_service.checks]]
  grace_period = '15s'
  interval = '30s'
  method = 'GET'
  timeout = '30s'
  path = '/health_check'
  protocol = 'http'

[[vm]]
  memory = '512mb'
  cpu_kind = 'shared'
  cpus = 1

[[statics]]
  guest_path = '/app/public'
  url_prefix = '/'

[[restart]]
  policy = 'always'
  retries = 5
  processes = ['app']

But the health check is not working and I get the following log:

$ fly logs -a name-of-the-app -i d8dd964a445568

Waiting for logs...

2024-08-12T17:31:20.836 app[d8dd964a445568] cdg [info] INFO Main child exited normally with code: 0

2024-08-12T17:31:20.850 app[d8dd964a445568] cdg [info] INFO Starting clean up.

2024-08-12T17:31:20.852 app[d8dd964a445568] cdg [info] WARN could not unmount /rootfs: EINVAL: Invalid argument

2024-08-12T17:31:20.853 app[d8dd964a445568] cdg [info] [ 469.347753] reboot: Restarting system

2024-08-12T17:31:25.830 app[d8dd964a445568] cdg [info] 2024-08-12T17:31:25.830440394 [01J53T9MGPSD8VDNFJ5MVMF5KZ:main] Running Firecracker v1.7.0

2024-08-12T17:31:26.192 app[d8dd964a445568] cdg [info] [ 0.283760] PCI: Fatal: No config space access function found

2024-08-12T17:31:26.553 app[d8dd964a445568] cdg [info] INFO Starting init (commit: 20f21dc5f)...

2024-08-12T17:31:26.591 app[d8dd964a445568] cdg [info] INFO starting statics vsock server

2024-08-12T17:31:26.591 app[d8dd964a445568] cdg [info] INFO Preparing to run: `/app/bin/docker-entrypoint ./bin/rails server` as rails

2024-08-12T17:31:26.600 app[d8dd964a445568] cdg [info] INFO [fly api proxy] listening at /.fly/api

2024-08-12T17:31:26.609 app[d8dd964a445568] cdg [info] 2024/08/12 17:31:26 INFO SSH listening listen_address=[fdaa:9:bbd9:a7b:87:c4bc:56b:2]:22 dns_server=[fdaa::3]:53

2024-08-12T17:31:26.614 runner[d8dd964a445568] cdg [info] Machine created and started in 16.881s

2024-08-12T17:31:27.694 health[d8dd964a445568] cdg [error] Health check on port 3000 has failed. Your app is not responding properly. Services exposed on ports [80, 443] will have intermittent failures until the health check passes.

2024-08-12T17:31:28.981 app[d8dd964a445568] cdg [info] => Booting Puma

2024-08-12T17:31:28.981 app[d8dd964a445568] cdg [info] => Rails 7.1.3.2 application starting in production

2024-08-12T17:31:28.981 app[d8dd964a445568] cdg [info] => Run `bin/rails server --help` for more startup options

2024-08-12T17:31:30.340 app[d8dd964a445568] cdg [info] Puma starting in single mode...

2024-08-12T17:31:30.340 app[d8dd964a445568] cdg [info] * Puma version: 6.4.2 (ruby 3.3.1-p55) ("The Eagle of Durango")

2024-08-12T17:31:30.340 app[d8dd964a445568] cdg [info] * Min threads: 5

2024-08-12T17:31:30.340 app[d8dd964a445568] cdg [info] * Max threads: 5

2024-08-12T17:31:30.340 app[d8dd964a445568] cdg [info] * Environment: production

2024-08-12T17:31:30.340 app[d8dd964a445568] cdg [info] * PID: 317

2024-08-12T17:31:30.340 app[d8dd964a445568] cdg [info] * Listening on http://0.0.0.0:3000

2024-08-12T17:31:30.368 app[d8dd964a445568] cdg [info] Use Ctrl-C to stop

2024-08-12T17:36:44.060 runner[d8dd964a445568] cdg [info] Pulling container image registry.fly.io/name-of-the-app:deployment-01J53TGGQC2JG9T9MR6X87VJ3Q

2024-08-12T17:36:44.656 runner[d8dd964a445568] cdg [info] Successfully prepared image registry.fly.io/name-of-the-app:deployment-01J53TGGQC2JG9T9MR6X87VJ3Q (595.305058ms)

2024-08-12T17:36:46.343 app[d8dd964a445568] cdg [info] INFO Sending signal SIGINT to main child process w/ PID 317

2024-08-12T17:36:46.351 app[d8dd964a445568] cdg [info] - Gracefully stopping, waiting for requests to finish

2024-08-12T17:36:46.351 app[d8dd964a445568] cdg [info] === puma shutdown: 2024-08-12 17:36:46 +0000 ===

2024-08-12T17:36:46.351 app[d8dd964a445568] cdg [info] - Goodbye!

2024-08-12T17:36:46.351 app[d8dd964a445568] cdg [info] Exiting

2024-08-12T17:36:46.910 health[d8dd964a445568] cdg [warn] Health check on port 3000 is in a 'warning' state. Your app may not be responding properly. Services exposed on ports [80, 443] may have intermittent failures until the health check passes.

2024-08-12T17:36:47.055 app[d8dd964a445568] cdg [info] INFO Main child exited normally with code: 0

2024-08-12T17:36:47.072 app[d8dd964a445568] cdg [info] INFO Starting clean up.

2024-08-12T17:36:47.076 app[d8dd964a445568] cdg [info] WARN could not unmount /rootfs: EINVAL: Invalid argument

2024-08-12T17:36:47.076 app[d8dd964a445568] cdg [info] [ 321.178358] reboot: Restarting system

2024-08-12T17:36:47.491 health[d8dd964a445568] cdg [error] Health check on port 3000 has failed. Your app is not responding properly. Services exposed on ports [80, 443] will have intermittent failures until the health check passes.

2024-08-12T17:36:53.021 app[d8dd964a445568] cdg [info] 2024-08-12T17:36:53.020719372 [01J53TKTZYCHD3NS0DDSBX8T0F:main] Running Firecracker v1.7.0

2024-08-12T17:36:53.413 app[d8dd964a445568] cdg [info] [ 0.293782] PCI: Fatal: No config space access function found

2024-08-12T17:36:53.797 app[d8dd964a445568] cdg [info] INFO Starting init (commit: 20f21dc5f)...

2024-08-12T17:36:53.841 app[d8dd964a445568] cdg [info] INFO starting statics vsock server

2024-08-12T17:36:53.842 app[d8dd964a445568] cdg [info] INFO Preparing to run: `/app/bin/docker-entrypoint ./bin/rails server` as rails

2024-08-12T17:36:53.849 app[d8dd964a445568] cdg [info] INFO [fly api proxy] listening at /.fly/api

2024-08-12T17:36:53.853 app[d8dd964a445568] cdg [info] 2024/08/12 17:36:53 INFO SSH listening listen_address=[fdaa:9:bbd9:a7b:87:c4bc:56b:2]:22 dns_server=[fdaa::3]:53

2024-08-12T17:36:53.886 runner[d8dd964a445568] cdg [info] Machine created and started in 9.841s

2024-08-12T17:36:56.536 app[d8dd964a445568] cdg [info] => Booting Puma

2024-08-12T17:36:56.536 app[d8dd964a445568] cdg [info] => Rails 7.1.3.2 application starting in production

2024-08-12T17:36:56.536 app[d8dd964a445568] cdg [info] => Run `bin/rails server --help` for more startup options

2024-08-12T17:36:58.160 app[d8dd964a445568] cdg [info] Puma starting in single mode...

2024-08-12T17:36:58.160 app[d8dd964a445568] cdg [info] * Puma version: 6.4.2 (ruby 3.3.1-p55) ("The Eagle of Durango")

2024-08-12T17:36:58.160 app[d8dd964a445568] cdg [info] * Min threads: 5

2024-08-12T17:36:58.160 app[d8dd964a445568] cdg [info] * Max threads: 5

2024-08-12T17:36:58.160 app[d8dd964a445568] cdg [info] * Environment: production

2024-08-12T17:36:58.160 app[d8dd964a445568] cdg [info] * PID: 317

2024-08-12T17:36:58.161 app[d8dd964a445568] cdg [info] * Listening on http://0.0.0.0:3000

2024-08-12T17:36:58.195 app[d8dd964a445568] cdg [info] Use Ctrl-C to stop

2024-08-12T17:43:14.772 proxy[d8dd964a445568] cdg [info] App name-of-the-app has excess capacity, autostopping machine d8dd964a445568. 1 out of 2 machines left running (region=cdg, process group=app)

2024-08-12T17:43:14.783 app[d8dd964a445568] cdg [info] INFO Sending signal SIGINT to main child process w/ PID 317

2024-08-12T17:43:14.789 app[d8dd964a445568] cdg [info] - Gracefully stopping, waiting for requests to finish

2024-08-12T17:43:14.789 app[d8dd964a445568] cdg [info] === puma shutdown: 2024-08-12 17:43:14 +0000 ===

2024-08-12T17:43:14.789 app[d8dd964a445568] cdg [info] - Goodbye!

2024-08-12T17:43:14.789 app[d8dd964a445568] cdg [info] Exiting

2024-08-12T17:43:15.341 app[d8dd964a445568] cdg [info] INFO Main child exited normally with code: 0

2024-08-12T17:43:15.358 app[d8dd964a445568] cdg [info] INFO Starting clean up.

2024-08-12T17:43:15.367 app[d8dd964a445568] cdg [info] WARN could not unmount /rootfs: EINVAL: Invalid argument

2024-08-12T17:43:15.370 app[d8dd964a445568] cdg [info] [ 382.265613] reboot: Restarting system

2024-08-12T17:43:22.422 runner[d8dd964a445568] cdg [info] Pulling container image registry.fly.io/name-of-the-app:deployment-01J53TWBPM0FYSWPKYE8XV20RB

2024-08-12T17:43:30.168 runner[d8dd964a445568] cdg [info] Successfully prepared image registry.fly.io/name-of-the-app:deployment-01J53TWBPM0FYSWPKYE8XV20RB (7.746180388s)

2024-08-12T17:43:30.903 runner[d8dd964a445568] cdg [info] Configuring firecracker

2024-08-12T17:47:15.194 runner[d8dd964a445568] cdg [info] Pulling container image registry.fly.io/name-of-the-app:deployment-01J53V3NTFN2DC7VQ5QFQNWS1Q

2024-08-12T17:47:23.629 runner[d8dd964a445568] cdg [info] Successfully prepared image registry.fly.io/name-of-the-app:deployment-01J53V3NTFN2DC7VQ5QFQNWS1Q (8.435798629s)

2024-08-12T17:47:24.480 runner[d8dd964a445568] cdg [info] Configuring firecracker

2024-08-12T18:04:58.401 runner[d8dd964a445568] cdg [info] Pulling container image registry.fly.io/name-of-the-app:deployment-01J53W43C94750NT2GQACVTYY8

2024-08-12T18:04:59.718 runner[d8dd964a445568] cdg [info] Successfully prepared image registry.fly.io/name-of-the-app:deployment-01J53W43C94750NT2GQACVTYY8 (1.316686931s)

2024-08-12T18:09:50.972 runner[d8dd964a445568] cdg [info] Pulling container image registry.fly.io/name-of-the-app:deployment-01J53WETB5S6K7M4M9EC5B6CPY

2024-08-12T18:09:52.550 runner[d8dd964a445568] cdg [info] Successfully prepared image registry.fly.io/name-of-the-app:deployment-01J53WETB5S6K7M4M9EC5B6CPY (1.57839655s)

2024-08-12T18:09:53.119 runner[d8dd964a445568] cdg [info] Configuring firecracker

2024-08-12T18:12:43.229 proxy[d8dd964a445568] cdg [info] Starting machine

2024-08-12T18:12:43.360 app[d8dd964a445568] cdg [info] 2024-08-12T18:12:43.360507413 [01J53WGFB4GMYSS6TXJ5PC3K7M:main] Running Firecracker v1.7.0

2024-08-12T18:12:43.737 app[d8dd964a445568] cdg [info] [ 0.282913] PCI: Fatal: No config space access function found

2024-08-12T18:12:44.107 app[d8dd964a445568] cdg [info] INFO Starting init (commit: 20f21dc5f)...

2024-08-12T18:12:44.151 app[d8dd964a445568] cdg [info] INFO starting statics vsock server

2024-08-12T18:12:44.152 app[d8dd964a445568] cdg [info] INFO Preparing to run: `/app/bin/docker-entrypoint ./bin/rails server` as rails

2024-08-12T18:12:44.159 app[d8dd964a445568] cdg [info] INFO [fly api proxy] listening at /.fly/api

2024-08-12T18:12:44.164 app[d8dd964a445568] cdg [info] 2024/08/12 18:12:44 INFO SSH listening listen_address=[fdaa:9:bbd9:a7b:87:c4bc:56b:2]:22 dns_server=[fdaa::3]:53

2024-08-12T18:12:44.224 runner[d8dd964a445568] cdg [info] Machine started in 990ms

2024-08-12T18:12:44.225 proxy[d8dd964a445568] cdg [info] machine started in 996.124496ms

2024-08-12T18:12:44.232 proxy[d8dd964a445568] cdg [info] machine became reachable in 6.002247ms

2024-08-12T18:12:44.238 proxy[d8dd964a445568] cdg [error] [PC01] instance refused connection. is your app listening on 0.0.0.0:3000? make sure it is not only listening on 127.0.0.1 (hint: look at your startup logs, servers often print the address they are listening on)

2024-08-12T18:12:46.139 app[d8dd964a445568] cdg [info] => Booting Puma

2024-08-12T18:12:46.139 app[d8dd964a445568] cdg [info] => Rails 7.1.3.2 application starting in production

2024-08-12T18:12:46.139 app[d8dd964a445568] cdg [info] => Run `bin/rails server --help` for more startup options

2024-08-12T18:12:47.489 app[d8dd964a445568] cdg [info] Puma starting in single mode...

2024-08-12T18:12:47.489 app[d8dd964a445568] cdg [info] * Puma version: 6.4.2 (ruby 3.3.1-p55) ("The Eagle of Durango")

2024-08-12T18:12:47.489 app[d8dd964a445568] cdg [info] * Min threads: 5

2024-08-12T18:12:47.489 app[d8dd964a445568] cdg [info] * Max threads: 5

2024-08-12T18:12:47.489 app[d8dd964a445568] cdg [info] * Environment: production

2024-08-12T18:12:47.489 app[d8dd964a445568] cdg [info] * PID: 317

2024-08-12T18:12:47.491 app[d8dd964a445568] cdg [info] * Listening on http://0.0.0.0:3000

2024-08-12T18:12:47.516 app[d8dd964a445568] cdg [info] Use Ctrl-C to stop

2024-08-12T18:14:09.663 proxy[d8dd964a445568] fra [error] [PR04] could not find a good candidate within 91 attempts at load balancing

Is anyone facing the same error?

2

From App not working to Questions / Help

3

I was able to fix this, I basically:

  1. Changed the app command to include the -b 0.0.0.0 flag
  2. Refactored how I was defining the HTTP check

Updated fly.toml below:

app = 'name-of-the-app'
primary_region = 'cdg'
console_command = '/app/bin/rails console'

[build]
  build-target = 'production'

[deploy]
  release_command = './bin/rails db:prepare'

[processes]
  app = './bin/rails server -b 0.0.0.0'
  sidekiq = 'bundle exec sidekiq'

[http_service]
  internal_port = 3000
  force_https = true
  auto_stop_machines = 'stop'
  auto_start_machines = true
  min_machines_running = 1
  processes = ['app']

[[vm]]
  memory = '512mb'
  cpu_kind = 'shared'
  cpus = 1

[[statics]]
  guest_path = '/app/public'
  url_prefix = '/'

[[restart]]
  policy = 'always'
  retries = 5
  processes = ['app']

[checks]
  [checks.app_http_check]
    grace_period = '30s'
    interval = '15s'
    method = 'GET'
    path = "/health_check"
    port = 3000
    timeout = '10s'
    type = 'http'
    tls_skip_verify = false
    [checks.app_http_check.headers]
      X-Forwarded-Proto = 'https'

  [checks.app_tcp_check]
    grace_period = '30s'
    interval = '15s'
    port = 3000
    timeout = '10s'
    type = 'tcp'
4

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.