Figuring out the best concurrency limits

How can I figure out the best [http_service.concurrency] connections limit?

I keep getting an error that all concurrent connections have been used up but i’m not sure how to figure out a good limit?

Can I basically increase the limit until I have memory problems? How do i know how much memory is free, is that what Firecracker Memory Usage shows?

Is anyone able to help with this?

hi @p-christ

You could try setting a higher hard_limit to avoid the concurrency errors while you work on understanding what your soft_limit should be to optimize load balancing between Machines. You’ll also probably want to adjust your settings to real-world traffic once the app is released or in use. It can be a bit of trial and error and really depends on your workload.

This post offers some further tips for figuring out the best settings for concurrency:

thanks but how do I know if i am running out of CPU or Memory?

I am assuming Firecracker Memory Usage is what i look at for memory? Do you know what i look at for CPU?

You can look at Load Average. But Grafana will give you more detail, you can see memory and cpu utilization side-by-side.

In Grafana, if you look at the Fly Instance dashboard, you can see metrics per Machine, including individual memory and CPU data, and Fly Instance > Proxy will show concurrency info.

Here’s some info about metrics: Metrics on · Fly Docs
Link to Grafana:

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.