I have a python server on fly.io where each request takes approx 5 secs (8 processes for concurrency) with the following toml config
[http_service]
internal_port = 9000
force_https = true
auto_stop_machines = false
auto_start_machines = false
min_machines_running = 1
processes = ['app']
[http_service.concurrency]
type = "requests"
soft_limit = 50000
hard_limit = 100000
Also increased ulimit in entrypoint.sh based on a previous answer here.
#!/bin/bash
if [ $UID -eq 0 ]; then
ulimit -n 500000
echo "executed ulimit"
fi
I keep getting this error when I fire 19000 requests (note that this is lesser than the soft limit) at once.
2024-06-24T13:24:47Z proxy[148e2907f03238] sin [warn]Dropped a connection destined for tcp/443 either due to a full backlog for your app or rate limiting. We'll only notify again every power of 10 (hint: check if your app is closing connections properly when it's done with them)
2024-06-24T13:24:47Z proxy[148e2907f03238] sin [warn]Dropped 10 connections destined for tcp/443 either due to a full backlog for your app or rate limiting. (hint: check if your app is closing connections properly when it's done with them)
2024-06-24T13:24:47Z proxy[148e2907f03238] sin [warn]Dropped 100 connections destined for tcp/443 either due to a full backlog for your app or rate limiting. (hint: check if your app is closing connections properly when it's done with them)
2024-06-24T13:24:48Z proxy[148e2907f03238] sin [warn]Dropped 1000 connections destined for tcp/443 either due to a full backlog for your app or rate limiting. (hint: check if your app is closing connections properly when it's done with them)
2024-06-24T13:24:49Z proxy[148e2907f03238] sin [warn]Dropped 10000 connections destined for tcp/443 either due to a full backlog for your app or rate limiting. (hint: check if your app is closing connections properly when it's done with them)
2024-06-24T13:24:49Z proxy[148e2907f03238] sin [warn]Dropped a connection destined for tcp/443 either due to a full backlog for your app or rate limiting. We'll only notify again every power of 10 (hint: check if your app is closing connections properly when it's done with them)
2024-06-24T13:24:49Z proxy[148e2907f03238] sin [warn]Dropped 10 connections destined for tcp/443 either due to a full backlog for your app or rate limiting. (hint: check if your app is closing connections properly when it's done with them)
What is this error ? Can I increase the rate limit somewhere ? I have also tried this with type=“connections” and in both REST and GRPC servers. Get the same error.