Load balancing with the concurrency soft limit parameter

1

Hello! We rely on the concurrency soft limit parameter to load balance traffic on our applications.

Right now we have 2 instances :

Instances
ID      	PROCESS	VERSION	REGION	DESIRED	STATUS 	HEALTH CHECKS     	RESTARTS	CREATED
df8615e8	app    	427    	cdg   	run    	running	2 total, 2 passing	0       	2022-02-21T08:43:15Z
f68e4fed	app    	427    	lhr(B)	run    	running	2 total, 2 passing	0       	2022-02-21T08:42:50Z

In our fly.toml we have :

[services.concurrency]
    hard_limit = 30000
    soft_limit = 100
    type = "connections"

But it seems that it doesn’t work anymore :

image
image2816Ă—694 129 KB

Have you made some changes recently on this?

2

When you say “doesn’t work anymore”, were you actually getting a better split across two regions with the soft limit?

We consider both soft limit and latency when load balancing. If most of your traffic is hitting cdg, it makes sense that most of the requests would go there as long as it’s below the hard limit.

In general, soft limits are more of a hint, and hard limits are the only limit that is strictly enforced.

Are you intentionally running in CDG and LHR? If not, you’ll probably get a better result by setting fly regions backup cdg.

1 Like