http service health check woes using Flask

Python
1

I’ve defined a http check section as per the docs:

[[services]]
  protocol = 'tcp'
  internal_port = 8080
  auto_stop_machines = 'suspend'
  auto_start_machines = true
  min_machines_running = 0
  processes = ['app']

  [[services.ports]]
    port = 80
    handlers = ['http']
    force_https = true

  [[services.ports]]
    port = 443
    handlers = ['tls', 'http']

  [services.concurrency]
    type = 'requests'
    hard_limit = 20
    soft_limit = 10

  [[services.tcp_checks]]
    interval = '15s'
    timeout = '2s'
    grace_period = '1s'

  [[services.http_checks]]
    type = "http"
    grace_period = "10s"
    interval = "30s"
    method = "GET"
    timeout = "5s"
    path = "/healthcheck"
    tls_skip_verify = false
    [services.http_checks.headers]

And I’ve defined a route in Flask:

    @app.route("/healthcheck", methods=["GET"])
    def healthcheck():
        return jsonify({"status": "OK"}, 200)

Testing locally, I get a 200 OK response from that route, but as soon as I enable the health check, my deploys begin warning me that health checks are failing. Running fly checks list:

 NAME                      | STATUS   | MACHINE        | LAST UPDATED | OUTPUT
----------------------------*----------*----------------*--------------*--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  bg_deployments_http       | critical | e286ee3efe5218 | 1s ago       | 302 FOUND
                            |          |                |              | <!doctype html>
                            |          |                |              | <html lang=en>
                            |          |                |              | <title>Redirecting...</title>
                            |          |                |              | <h1>Redirecting...</h1>
                            |          |                |              | <p>You should be redirected automatically to the target URL: <a href="https://172.19.2.18:8080/healthcheck">https://172.19.2.18:8080/healthcheck</a>. If not, click the link.
----------------------------*----------*----------------*--------------*--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  bg_deployments_tcp        | passing  | e286ee3efe5218 | 16s ago      | Success
----------------------------*----------*----------------*--------------*--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  servicecheck-00-tcp-8080  | passing  | e286ee3efe5218 | 20s ago      | Success
----------------------------*----------*----------------*--------------*--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  servicecheck-01-http-8080 | critical | e286ee3efe5218 | 3s ago       | 302 FOUND
                            |          |                |              | <!doctype html>
                            |          |                |              | <html lang=en>
                            |          |                |              | <title>Redirecting...</title>
                            |          |                |              | <h1>Redirecting...</h1>
                            |          |                |              | <p>You should be redirected automatically to the target URL: <a href="https://172.19.2.18:8080/healthcheck">https://172.19.2.18:8080/healthcheck</a>. If not, click the link.
----------------------------*----------*----------------*--------------*----------------------------------------------------------------------------------------------------------------------------

I assume those redirects aren’t correct? But I have no idea what’s going on.

3

These redirects come from your Flask application, it’s not something the Fly platform or proxy are doing. The redirects are confusing the health checker, which expects a 2xx response in order to consider the health check successful.

Sounds like you may be using flask-sslify or somehow rewriting incoming requests to be https instead of http. In most cases your Flask app shouldn’t handle this; have your app returning HTTP only instead. If you have a specific need to do things this way I’d love to hear why!

  • Daniel
4

That was the problem: I’m using GitHub - GoogleCloudPlatform/flask-talisman: HTTP security headers for Flask to build my CSP, and of course by default it sends a header to redirect all http traffic to https. Disabling that for the health check route fixes it. Thanks for the pointer!

5

If you use force_https = true in your fly.toml (see here) the Fly proxy takes care of this for you, so you can disable it globally for your app.

  • Daniel
1 Like
6

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.