Fly app does not start ("net/http: request canceled" error) with http/2 server

[lodybo]

I have an app running on Fly (two machines) for which I’ve recently switched my Express server for an HTTP/2 server with spdy. The reason for which was that I needed the multiplexing feature in my app.

I tried to deploy my app, but all the deployments fail because of net/http: request canceled errors. I’ve tried changing the protocol of my healthcheck configuration but nothing seems to be working. I can see in the dashboard that the TCP check on port 8080 succeeds, but the HTTP check on port 8080 fails with “connect: connection refused”.

I’m now not sure what steps I’d need to take in order to address this issue, because it seems like the machine will has booted up, but it cannot be reached.

For reference, this is my config file:

# fly.toml app configuration file generated for weddingfest on 2023-06-23T15:19:48+02:00
#
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#

app = "weddingfest"
primary_region = "ams"
kill_signal = "SIGINT"
kill_timeout = "5s"

[experimental]
  auto_rollback = true

[deploy]
  release_command = "npm run release"

[env]
  PORT = "8080"
  PRIMARY_REGION = "ams"

[mounts]
  source = "weddingfest_data"
  destination = "/weddingfest/images"

[[services]]
  protocol = "tcp"
  internal_port = 8080
  processes = ["app"]

  [[services.ports]]
    port = 80
    handlers = ["http"]
    force_https = true

  [[services.ports]]
    port = 443
    handlers = ["tls", "http"]


  [services.concurrency]
    type = "connections"
    hard_limit = 25
    soft_limit = 20

  [[services.tcp_checks]]
    interval = "15s"
    timeout = "2s"
    grace_period = "1s"
    restart_limit = 0

  [[services.http_checks]]
    interval = "10s"
    timeout = "2s"
    grace_period = "5s"
    restart_limit = 0
    method = "get"
    path = "/healthcheck"
    protocol = "http"

A few notes:

• In the [[services.http_checks]] block I also tried the protocol = "https" option in combination with tls_skip_verify = false, but that yielded the same error.
• Before I changed the server package, the app ran and could be connected to. I only switched the server package but kept the rest of the server code the same.

[kwaw]

hey @lodybo, my initial suspicion is that this has something to do with the configuration of spyd. unfortunately I have 0 experience with it. does your service run well elsewhere?

health checks don’t go through fly proxy, consul tries to directly ping the vm.

[rubys]

@lodybo

handlers = ["tls", "http"]

You don’t want both fly and spdy handling tls. Have you tried configuring spdy to use plain: true, ssl: false? See Is it possible to use SPDY without HTTPS? · Issue #103 · spdy-http2/node-spdy · GitHub

[lodybo]

Thanks, @kwaw and @rubys. I went and double checked a few things based on your comments:

• My healthcheck did not work locally since I use a self-signed certificate during development. That lead me down the path of configuring either a workaround for local development since Fly will try to directly ping the vm, or to see if I could develop locally without HTTPS.
• plain: true, ssl: false only seemed to work on the client. The ssl option didn’t exist on the server.
• Most importantly: spdy seems to be unmaintained, the last commit was pushed in 2020.

So I switched out the spdy package and went back to the “old” server. I didn’t encounter many performance issues with multiplexing and was therefore overthinking things. Your comments helped me reflect on this with a new perspective