App listening on tcp://0.0.0.0:3000, but healthcheck fails on port 3000

Questions / Help
1

Hi, my app was running without issues before, having the last successful release in 2023-05-02.
The day before yesterday I made some changes and tried to do a deploy. After the deploy happened, the app is down, and I cannot find any logs related to the error.

Here is what I can see in the metrics / logs:

2023-10-02T20:12:34.117 app[32874244f63758] otp [info] [ 1620.145340] reboot: Restarting system

2023-10-02T20:12:34.548 app[32874244f63758] otp [info] [ 0.040233] PCI: Fatal: No config space access function found

2023-10-02T20:12:34.776 app[32874244f63758] otp [info] INFO Starting init (commit: 5d9c42f)...

2023-10-02T20:12:34.803 app[32874244f63758] otp [info] INFO Preparing to run: `bundle exec rails server -b 0.0.0.0 -p 3000` as root

2023-10-02T20:12:34.811 app[32874244f63758] otp [info] INFO [fly api proxy] listening at /.fly/api

2023-10-02T20:12:34.819 app[32874244f63758] otp [info] 2023/10/02 20:12:34 listening on [fdaa:2:38b:a7b:1aa:bd0e:8608:2]:22 (DNS: [fdaa::3]:53)

2023-10-02T20:12:36.222 app[32874244f63758] otp [info] => Booting Puma

2023-10-02T20:12:36.222 app[32874244f63758] otp [info] => Rails 7.0.4.3 application starting in production

2023-10-02T20:12:36.222 app[32874244f63758] otp [info] => Run `bin/rails server --help` for more startup options

2023-10-02T20:12:37.433 app[32874244f63758] otp [info] Puma starting in single mode...

2023-10-02T20:12:37.433 app[32874244f63758] otp [info] * Version 4.3.12 (ruby 3.1.2-p20), codename: Mysterious Traveller

2023-10-02T20:12:37.433 app[32874244f63758] otp [info] * Min threads: 5, max threads: 5

2023-10-02T20:12:37.433 app[32874244f63758] otp [info] * Environment: production

2023-10-02T20:12:37.433 app[32874244f63758] otp [info] * Listening on tcp://0.0.0.0:3000

2023-10-02T20:12:37.434 app[32874244f63758] otp [info] Use Ctrl-C to stop

2023-10-02T20:12:41.823 health[32874244f63758] otp [error] Health check on port 3000 has failed. Your app is not responding properly. Services exposed on ports [80, 443] will have intermittent failures until the health check passes.

Here is my fly.toml file:

# fly.toml file generated for <my-app-name> on 2023-04-18T14:52:29+03:00

app = "<my-app-name>"
primary_region = "otp"
kill_signal = "SIGTERM"
swap_size_mb = 512

[env]
  RAILS_ENV = "production"

[http_service]
  processes = ["web"]
  internal_port = 3000
  protocol = "tcp"
  force_https = true
  min_machine_running = 1
  [http_service.concurrency]
    type = "requests"
    soft_limit = 20
    hard_limit = 45
  [[http_service.checks]]

[processes]
  web = "bundle exec rails server -b 0.0.0.0 -p 3000"
  sidekiq = "bundle exec sidekiq -q production_default"

I can see that there is a critical check:

> flyctl checks list                                                                                                                                                                                                         08:46:42
Health Checks for gold-itp
  NAME                      | STATUS   | MACHINE        | LAST UPDATED | OUTPUT
----------------------------*----------*----------------*--------------*------------------------------
  servicecheck-00-http-3000 | critical | 32874244f63758 | 9h35m ago    | 403 Forbidden
----------------------------*----------*----------------*--------------*------------------------------

but I cannot find a good reason for this critical error on the check. Can someone please take a look and help me out?

Thanks in advance

2

I see you have [[http_service.checks]] in fly.toml, but it doesn’t look like you have any checks set up. What happens if you remove [[http_service.checks]] and run another deploy?

3

hi @brian
The [[http_service.checks]] was present because it was required the first time I created the config. Seems like it is not required anymore, and without it the deployment succeeded. Thanks for pointing it out.

4

Generally speaking, setting health checks is probably a good idea. In this case, not specifying the parameters for the health check probably creates a health check with default options, which may not work for your app.

5

I tried setting the parameters too, and it still failed. Without the healthcheck everything is working as expected. I will try to experiment with a TCP health check instead of the HTTP one

6

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.