How should I set the correct hard_limit and soft_limit number?

2

A sample configuration:

  [services.concurrency]
    type = "requests"
    hard_limit = 25
    soft_limit = 20

More info:

Load Balancing - Web Service

Reference Definitions

Troubleshooting - Not Enough Connections

A couple of things you can do to determine the appropriate hard_limit and soft_limit values for your Remix site (you’ll need to consider your application’s performance characteristics and resource usage)

Here’s one approach to help you find suitable values:

  1. Benchmark your application: Run performance tests on your application to determine how many concurrent requests it can handle without degrading performance or running out of resources (CPU, memory). You can use tools like Apache JMeter, Locust, or k6 for load testing.

  2. Set initial values: Based on your benchmark results, set initial hard_limit and soft_limit values in your fly.toml file. The hard_limit should be the maximum number of concurrent requests your application can handle without issues, while the soft_limit should be slightly lower to provide some buffer for Fly.io to shift load to other instances before an instance becomes overwhelmed.

  [services.concurrency]
    type = "requests"
    hard_limit = <your_hard_limit>
    soft_limit = <your_soft_limit>
  1. Monitor and adjust: After deploying your application with the initial concurrency settings, monitor its performance and resource usage. If you notice that your application is struggling to handle the load or is using too many resources, you can adjust the hard_limit and soft_limit values accordingly.

Keep in mind that finding the optimal concurrency settings may require some trial and error. It’s essential to continuously monitor your application’s performance and make adjustments as needed to ensure a smooth user experience.

Additionally, consider scaling your application horizontally by deploying more instances across different regions to distribute the load and improve performance.

3 Likes