Warning about hard limit on websocket app

1

Hi,

I’m running into an issue with autoscaling on my app (running apps v1 still). I’m receiving this warning and I’m confusing about why it wouldn’t be already scaling my resources?

[warn] Instance <> reached hard limit of 200 concurrent connections. This usually indicates your app is not closing connections properly or is not closing them fast enough for the traffic levels it is handling. Scaling resources, number of instances or increasing your hard limit might help.

My application is a websocket app so it’s got long-running connections but the number of instances that are running are below the max count that I have set. I thought that Apps was supposed to handle scaling based on the connection limit that I set? I’ve also got multiple regions set up, am I just going to get this warning until the connections drop and in the meantime the traffic will be directed to another region?

Any insight would be appreciated :slight_smile:

2

Are you able to provide the configuration (fly.toml) for the app?

If you haven’t already, you may want to try increasing the concurrency limits: https://fly.io/docs/reference/configuration/#services-concurrency.

3

This is my fly.toml (the app name is set via CLI on deployment).

kill_signal = "SIGTERM"
kill_timeout = 15
processes = []

[env]
  PORT = "8080"

[experimental]
  allowed_public_ports = []
  auto_rollback = true

[deploy]
  strategy = "bluegreen"

[[services]]
  http_checks = []
  internal_port = 8080
  processes = ["app"]
  protocol = "tcp"
  script_checks = []
  [services.concurrency]
    hard_limit = 200
    soft_limit = 100
    type = "connections"

  [[services.ports]]
    force_https = true
    handlers = ["http"]
    port = 80

  [[services.ports]]
    handlers = ["tls", "http"]
    port = 443

  [[services.tcp_checks]]
    grace_period = "15s"
    interval = "15s"
    restart_limit = 2
    timeout = "5s"

I can increase the hard_limit but isn’t this limit how Nomad determines adding new instances? Why wouldn’t it be doing this? The docs mention this on hard limits:

For Nomad apps only, the system will bring up another instance if the autoscaling policy supports doing so.

I doubled checked the policy on my app and autoscaling is enabled.

Edit: Guess the docs answered my own question about why it may not scale out:

Autoscaling is a horizontal scaling service based on a global pool of instances where the application can be run. The system will create at least the minimum number of application instances. The autoscaler will then create instances (scale-out) as total application load exceeds the soft_limit * [instance count] threshold, up to the max count. It will also remove instances (scale-in) when the max application load over a 10-minute window dips below the same threshold, down to the min count. The 10-minute scale-in window allows for rapid scale-out with slightly more relaxed scale-in.

4

The Apps v2 “autoscaling” is a lot more sane here. It’s not quite autoscaling the same as Apps v1 autoscaling is - we won’t create new instances (named Machines in apps v2) - but if you create as many machines as you think you’ll need, and set them to automatically start/stop, we’ll start an additional instance right in the HTTP service handler when the currently running instances are at capacity. Here’s the docs for this: Automatically Stop and Start Machines · Fly Docs

6

Thanks for the link. I’d like to move to Apps V2 soon, just waiting on being able to use the bluegreen deployment strategy and being able to set the restart policy for the machine on the toml.

7

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.