How do you troubleshoot http_checks?

[Alex_Piechowski]

I have a health check, but the deployment with the health check won’t go through because the health check is failing

Here is my check:

  [[services.http_checks]]
    interval = 10000
    grace_period = "30s"
    method = "get"
    path = "/"
    protocol = "http"
    timeout = 2000
    tls_skip_verify = false

The deploy fails without any notion of why the health check is failing.

[charsleysa]

Hi @Alex_Piechowski

Does the healthcheck work locally when testing with something like Postman or curl?

[FrequentFlyer]

In addition to @charsleysa’s suggestion above…

One could program a GPT-3 bot to respond to posts here with Fly folks’ answers & suggestions

@kurt

When a deployment fails, the first step is to look at a failed VM and see what you can figure out.
…
To see the specific VM status, run fly status --all to get a list of VMs.
Find one with status failed , then run fly vm status <id> .
This will give you a lot more information.
Make sure you check the exit code.
If it’s 0 it means health check failures, if it’s not zero it’s some issue crashing the process.

Also try setting tls_skip_verify = true.
Don’t need TLS for an HTTP check.

Then, if you’re still stick in a redirect loop of some sort for the check, this should help:

@kurt

This is most likely because your health check is responding with a redirect to https://

Try just removing the health check entirely, this will make it default to a TCP connection check.

If that works, you’ll need to modify your app to accept a health check URL that doesn’t perform a redirect. Health checks have to return a 200 response to count.

[Alex_Piechowski]

Hmm, works with curl @ localhost:8080
I’ve tried all of the variations listed in the other postings
The issue with the “troubleshoot failure” is that since the deploy never actually goes through, it never shows up in that list

[kurt]

What kind of app is this? Usually when. health checks fail it’s because it’s not listening on 0.0.0.0. Some frameworks default to 127.0.0.1 only.

[Alex_Piechowski]

This is a Lucky (Crystal) app. https://github.com/grepsedawk/bonspiel.studio
The app works fine with the http_check removed, so I don’t believe that could be caused by a binding issue correct?

I would still like the http_check for securities around the application being up per my standards

[jerome]

Are you setting the HOST environment variable anywhere? I don’t know what the default is, but it should be 0.0.0.0.

[Alex_Piechowski]

I shouldn’t need to, I am directly binding to 0.0.0.0 https://github.com/grepsedawk/bonspiel.studio/blob/c8468cafbd209b78cb80af412f5fe6b28dcd835c/config/server.cr#L8
Is it a requirement to fly.io to set HOST=0.0.0.0 ?

[jerome]

Ah, I missed that. Though I dabbled with Crystal, I’m not entirely familiar with Lucky.

You probably don’t have to use a HOST env var.

You can fly ssh console into your app and maybe check what addresses it’s listening on:

• ip a s eth0
• Take note of the 172.19.x.x IP
• curl http://172.19.x.x:8080/ping

(All from within your SSH session into your instance)

You can do whatever you want in there! (e.g. install other diagnostic tools)

[Alex_Piechowski]

Yeah I’ve been playing around in the ssh console quite a bit already.

Turns out a https forwarded for header was the trick:

Fix http status checks