Inbound connections to 80 & 443 get reset on IPv6 only app

1

This is my first post, so please forgive any breaches of community etiquette!

I have a web app with a TOML config that looks like this:

[http_service]
  internal_port = 80
  force_https = true

[checks]
  [checks.http_check]
    port = 80
    grace_period = "30s"
    interval = "15s"
    restart_limit = 4
    timeout = "2s"
	method = "GET"
	path = "/check"

The app only has an IPv6 address for the moment, I will add a v4 address when I’m ready to launch. I have an SSL certificate issued for the HTTP transport.

I am experiencing an error similar to the one described in this thread:

Any connection to the app over port 80 or 443 gets reset on the other end.

My app is fine: when I SSH into the VM I can reach the internal port. The Dashboard reports all the health checks as passing, however, my app logs all requests, and I actually don’t see any requests for the /check URL in the logs, so I am not sure if that is working properly. Redeploying the app hasn’t helped, but I haven’t tried destroying and recreating yet.

AFAIK everything on my side is fine, so at this point my assumption is that it’s a problem with Fly. Can someone else sense check this?

Thanks!

2

Hi @Gavin,

Is this a publicly accessible app or internal only? If it’s a public then you’re missing the services section of your configuration.

See here App Configuration (fly.toml) · Fly Docs

EDIT: you have http_service section so this advice is misleading

3

The http_service expands to the right services block.

Do you see anything with fly logs? The most common problem is a process listening on the wrong IP address. Binding to 127.0.0.1 won’t work, the simplest fix is usually to listen on 0.0.0.0.

4

I added the [services] section to the TOML file and the app became reachable. When I then removed the [http_services] section it stopped working, so it seems like it needs both sections to be present to work. That doesn’t seem match up to either @charsleysa’s advice or @kurt’s (and the documentation).

Anyway, my app seems to be working now but I wanted to flag the mismatch between the documentation and the observed behaviour.

5

Hi @Gavin! Glad you got your app working!

This does not seem right, though. [http_services] gets expanded to a [[services]] block behind the scenes. It should make your fly.toml simpler, not more complicated.

If you can share your whole config, with any sensitive data removed, it may become clearer what’s going on.

6

Oh, another possibility, maybe: is it possible that you encountered this more recent IPV6 bug, and the change to your config is a red herring?

7

I don’t think my issue is related to that one as I don’t get the “Fly API error” messages.

I re-ran my tests and the app works with only the [[services]] section, but not when only the [[http_service]] section is present.

Here is the complete redacted TOML file:

app = "redacted"
kill_signal = "SIGINT"
kill_timeout = 5
primary_region = "ams"
processes = []

[experimental]
  cmd = ["redacted"]

[build]
  dockerfile = "../../Dockerfile"

[build.args]
  BRANCH="main"

[deploy]
  strategy = "rolling"

[env]
	# redacted

[[services]]
  internal_port = 80
  protocol = "tcp"

  [services.concurrency]
    type = "requests"
    hard_limit = 128
    soft_limit = 64

  [[services.ports]]
    port = 80
	handlers = ["http"]

  [[services.ports]]
    port = 443
	handlers = ["tls", "http"]

  [[services.tcp_checks]]
    grace_period = "30s"
    interval = "15s"
    restart_limit = 4
    timeout = "2s"

[http_service]
  internal_port = 80
  force_https = true

  [http_service.concurrency]
    type = "requests"
    soft_limit = 128
    hard_limit = 64

[checks]
  [checks.http_check]
    port = 80
    grace_period = "30s"
    interval = "15s"
    restart_limit = 4
    timeout = "2s"
	method = "GET"
	path = "/check"
8

Hi @Gavin! Is your app running on the Nomad platform? The http_service section works only for v2 apps (Machines platform).

I’m sorry you ran into this; it looks like our docs don’t mention this limitation, and flyctl deploy appears to silently ignore the section!

9

@MatthewIngwersen yes, my app is running on Nomad.

10

@MatthewIngwersen by the way, my app is a new(ish) one that I only created a couple of weeks ago, using fly launch. I wasn’t aware that using fly launch would create an app on a “legacy” platform - that’s not in the documentation IIRC.

11

Thanks for the info @Gavin!

The http_service docs have been updated to note that it’s for Apps v2 only, and flyctl (with v0.0.500, released today) will now emit a warning if http_service is used for a Nomad app. (We might change this to an error in the future, but for now we want to be careful not to break anybody’s pipelines.)

Sorry about the fly launch confusion! New users get Apps v2 by default now, but this change is very recent and happened after you deployed your app. If you want to opt your existing organization(s) into Apps v2-by-default, you can run fly orgs apps-v2 default-on <org-slug>. You can also try it out for a single app by using --machines with fly apps create or --force-machines with fly launch and fly deploy.

1 Like
12

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.