services.http_checks doesn't work

1

I can’t get the services.http_checks to work and I don’t understand why. When deploying, the check would always fail with this error:

Error: failed to update machine 683d56ea707218: Unrecoverable error: timeout reached waiting for health checks to pass for machine 683d56ea707218: failed to get VM 683d56ea707218: Get "https://api.machines.dev/v1/apps/xxx/machines/683d56ea707218": net/http: request canceled

and from the Live Logs page, it says:

Check                       Status      Output                      Updated
servicecheck-00-http-8080 	warning 	waiting for status update 	2024-09-19 16:05:48

The app is running fine without error and I can access the web page.

This is my fly.toml:

app = 'xxx'
primary_region = 'yyz'
kill_signal = 'SIGINT'
kill_timeout = '5s'

[experimental]
  auto_rollback = true

[build]
  builder = 'heroku/builder:22'

[env]
  PORT = '8080'

[http_service]
  internal_port = 8080
  force_https = true
  auto_stop_machines = 'off'
  auto_start_machines = false
  min_machines_running = 1
  processes = ['app']

[[services]]
  protocol = 'tcp'
  internal_port = 8080
  processes = ['app']

  [[services.ports]]
    port = 80
    handlers = ['http']
    force_https = true

  [[services.ports]]
    port = 443
    handlers = ['tls', 'http']

  [services.concurrency]
    type = 'connections'
    hard_limit = 25
    soft_limit = 20

  [[services.http_checks]]
    interval = '10s'
    timeout = '2s'
    grace_period = '5s'
    method = 'get'
    path = '/'
    protocol = 'https'
    tls_skip_verify = false

[[vm]]
  size = 'shared-cpu-1x'
2

Hi… You have an overlap in the [http_service] and [[services]] definitions, which often confuses the Fly.io infrastructure. I’d suggest merging the former into the latter.

(The protocol may need tweaking as well, but I don’t do much with health checks, myself.)

Hope this helps a little!

3

Added proxy

4

Thank you, I’ve updated the file to the following:

app = 'xxx'
primary_region = 'yyz'
kill_signal = 'SIGINT'
kill_timeout = '5s'

[experimental]
  auto_rollback = true

[build]
  builder = 'heroku/builder:22'

[env]
  PORT = '8080'

[http_service]
  internal_port = 8080
  force_https = true
  auto_stop_machines = 'off'
  auto_start_machines = false
  min_machines_running = 1
  processes = ['app']
  [http_service.concurrency]
    type = "requests"
    soft_limit = 200
    hard_limit = 250
  [[http_service.checks]]
    grace_period = "5s"
    interval = "10s"
    method = "GET"
    timeout = "2s"
    path = "/"
    protocol = 'https'

[[vm]]
  size = 'shared-cpu-1x'

The monitoring page now shows these messages:


Health check on port 8080 has failed. Your app is not responding properly. Services exposed on ports [80, 443] will have intermittent failures until the health check passes.
        
Health check on port 8080 is in a 'warning' state. Your app may not be responding properly. Services exposed on ports [80, 443] may have intermittent failures until the health check passes.
5

Hm… This looks similar to the following older post:

I think you really only want protocol = 'https' in the unusual case of when your own web app is handling SSL itself (instead of letting the Fly edge proxy handle it).

(A lot of the settings in fly.toml look at things from the Fly edge proxy’s perspective.)

6

Thank you, that seems to work! Instead of removing that protocol = "https" I changed it to protocol = "http" and removed the force_https = true line.

1 Like
7

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.