We now have custom Machine based health checks!

You can now have custom health checks in your Fly app (called Machine health checks)! These let you check that your app is functional in a more thorough way than tcp_checks and http_checks. As an example:

  [[services.machine_checks]]
    image = "curlimages/curl"
    entrypoint = ["/bin/sh", "-c"]
    command = ["curl", "$FLY_TEST_MACHINE_IP", "|", "grep", "Hello, World!"]

The above code, when added to fly.toml, will spawn a Machine to make sure that things are working before passing the deploy. The behavior varies from deployment strategy to strategy, but generally:

  • canary: we spawn a single test Machine to test the canary machine
  • rolling (default): we spawn a test Machine for each machine we’re deploying to

:warning: This type of check works differently than other health checks! The other health checks and machine_checks both check that your app is working on deploy. However, Machine checks do not continuously check your app or affect routing.
For those who don’t know, we route requests away from machine failing health checks. Machine checks don’t do that as of writing, mostly to avoid complexity in the implementation. However, if there’s enough of a desire, it absolutely could be added to flyctl.

2 Likes

This is nice and could be really useful for event-driven workers. Thanks for this great update! :tada:

Since you referenced the intended behavior of current health checks that will route requests away from failing machines, I’ve actually posted a question a few hours ago about this not working as intended. Could it be related to this change?

1 Like

Thank you! And likely not, since this feature didn’t make any changes to the proxy.