I have an envoy proxy in front of a Next.js application and a Go server so that they appear from the same origin. The envoy proxy is routing properly but the number of applications isn’t increasing and it’s overloading just one server (specifically with next.js).
How does Fly Proxy load balance between the different machines? Does it route all traffic coming from the same source to the same machine?
Hi… Machine-to-Machine load balancing and auto-scaling are only done on the Flycast network, and not on the .internal addresses. That’s the main thing that trips people up.
If you haven’t already, try assigning Flycast addresses to the Next.js and Go applications—and then route to port 80 on .flycast instead of port 8080 on .internal.
Hope this helps!
Added autoscaling, machines, proxy
I switched that over but the upstream peers are not scaling out still. auto_stop_machines and auto_start_machines are both set to true.
Hm… Maybe post the Next.js application’s full fly.toml… Perhaps the concurrency settings need to be tweaked.
https://fly.io/docs/reference/configuration/#http_service-concurrency
Also, how many of those machines do you currently have, and in which regions?
(The proxy won’t create new ones, only start and stop existing.)
Here is the toml. I have 5 machines in den and 5 in sea. Only one will fire up when going through the envoy proxy.
app = 'my-web-app'
primary_region = 'den'
[build]
[http_service]
internal_port = 3000
force_https = false
auto_stop_machines = true
auto_start_machines = true
min_machines_running = 0
processes = ['app']
[[vm]]
memory = '512mb'
cpu_kind = 'shared'
cpus = 1
Thanks for the additional details… I think you want an explicit [http_service.concurrency] section for this, tuned to the limits of your specific servers.
You can verify the current settings via fly m status -d, under "concurrency".
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.