Accessing one instance reaches the hard limit for all - why?

I have scaled an app to 5 computers. I want each connection to use a separate server.

[services.concurrency]
    hard_limit = 1
    soft_limit = 1
    type = "requests"`

However, when I open only one connection, all 5 reach the hard limit. What can cause this?

 2022-09-14T13:20:29.879 proxy[89bcb94f] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.329 proxy[89bcb94f] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.339 proxy[89bcb94f] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.362 proxy[89bcb94f] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.363 proxy[bce7509b] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.368 proxy[d2e6b68d] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.385 proxy[ccd1e775] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.390 proxy[89bcb94f] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.387 proxy[bce7509b] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.414 proxy[d2e6b68d] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.413 proxy[ccd1e775] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.417 proxy[a383c5a0] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.419 proxy[ccd1e775] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.423 proxy[bce7509b] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.423 proxy[ccd1e775] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.423 proxy[d2e6b68d] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.428 proxy[ccd1e775] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.436 proxy[d2e6b68d] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.436 proxy[ccd1e775] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.442 proxy[a383c5a0] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.459 proxy[89bcb94f] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.482 proxy[bce7509b] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.492 proxy[a383c5a0] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.494 proxy[bce7509b] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.508 proxy[bce7509b] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.533 proxy[bce7509b] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.537 proxy[d2e6b68d] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.553 proxy[bce7509b] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.568 proxy[89bcb94f] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.583 proxy[d2e6b68d] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.597 proxy[ccd1e775] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.645 proxy[89bcb94f] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.768 proxy[89bcb94f] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.777 proxy[ccd1e775] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.899 proxy[89bcb94f] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:30.926 proxy[bce7509b] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:31.311 proxy[89bcb94f] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:31.486 proxy[89bcb94f] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:31.511 proxy[d2e6b68d] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:31.514 proxy[a383c5a0] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:31.644 proxy[89bcb94f] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:31.954 proxy[89bcb94f] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:32.035 proxy[89bcb94f] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:32.044 proxy[89bcb94f] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:32.060 proxy[ccd1e775] fra [warn] Instance reached connections hard limit of 1

2022-09-14T13:20:32.073 proxy[ccd1e775] fra [warn] Instance reached connections hard limit of 1 ```
1 Like

Are you able to see when your VMs accept the connection that you opened? This message happens when we send the request to a different VM, or queue it. So if you’re able to tell when your connections are being accepted by the backends, that might help figure out what’s going on.

1 Like

fyi in case you didn’t know: a (tcp) connection may contain multiple (http) requests. If you want to load balance per connection, then change the entry to type = "connection".

Where can i see that? Aren’t these the time codes that are near the warning? All those appear just after i open a connection.

Thanks. This specific error persists with both options, but thanks for the explanation.

1 Like

BTW, app name : genitemp2

Thanks for sending that over! So for this app and its autoscaling there are a couple of things to keep in mind:

  • this is the message we log when fly-proxy attempts to send a request to your app instance, but the app instance’s host knows that the hard_limit has already been reached.
  • when that happens, we log the hard limit of 1 message and tell the proxy to retry. The retries are fast enough that it’s conceivable for one request to trigger multiple hard limit of 1 messages
  • For lower limits, you will more frequently see the case that proxy isn’t aware that the limit was reached for a given vm (since the limit is reached faster)
  • This app is reachable over the public internet, random connections from which may also impact the testing

All this taken together suggests that it’s important to also have app logs for when the connections were received by a given instance, should you need to use a lower hard_limit. The goal is to make sure that individual VMs only accept connections up to the hard_limit, so in testing you’d want to measure there.

1 Like

Technically, I do not use autoscale, but simply count scale to 5 different servers.

But 1 request triggers multiple hard limit of 1 messages on all 5 instances. How can this occur?

Your app instances’ hosts tell the proxy if the hard_limit is reached. The proxy will guess.

So you’d see this (1 request triggering all 5 app instances’ hard_limit message) in cases where the proxy thinks that the app instance is available to accept connections/requests but the app instance’s host does not. The proxy can also retry with other instances.

I have downscaled the app, and now it fails to start. The very moment I try to connect to it, I get

2022-09-14T17:37:26.823 proxy[c6da36fa] fra [warn] Instance reached connections hard limit of 1

2022-09-14T17:37:29.323 proxy[c6da36fa] fra [warn] Instance reached connections hard limit of 1

I have set

  [services.concurrency]
    hard_limit = 1
    soft_limit = 1
    type = "connections"

Certainly not a memory problem ( I have increased the memory).

Why can this occur? What can I try?