Why is my App not Scaling Down?

Questions / Help
1

I’m not seeing my apps machines scale down even though I have auto_stop_machines = “suspend” and a soft_limit set. Ive tested adjusting soft_limit to a high number and I am not seeing fly suspend my machines.

From my analytics, should I be looking at App Concurrency to see what soft_limit should be set to?

Below is my toml file.

app = "app"
primary_region = "ewr"
kill_signal = "SIGINT"
kill_timeout = "5s"

[experimental]
auto_rollback = true

[[mounts]]
source = "app_data"
destination = "/data"
processes = ["app"]

[[services]]
protocol = "tcp"
internal_port = 80
auto_start_machines = true
auto_stop_machines = "suspend"
min_machines_running = 1

[[services.ports]]
port = 80
handlers = ["proxy_proto"]
[services.ports.proxy_proto_options]
version = "v2"
[services.concurrency]
type = "connections"
hard_limit = 12000
soft_limit = 8000

[[services.tcp_checks]]
interval = "15s"
timeout = "5s"
grace_period = "1s"
restart_limit = 0

[[services]]
protocol = "tcp"
internal_port = 443
auto_start_machines = true
auto_stop_machines = "suspend"
min_machines_running = 1

[[services.ports]]
port = 443
handlers = ["proxy_proto"]
[services.ports.proxy_proto_options]
version = "v2"
[services.concurrency]
type = "connections"
hard_limit = 12000
soft_limit = 8000
2

Probably this

3

What is wrong with this? I don’t want the service to go offline. There will always be traffic to this application.

4

I’d recommend reading Autostop/autostart Machines · Fly Docs

5

Thanks for the tip. I have read the docs and added the values that should cause the app to scale down and up as needed but it doesn’t appear to be working. Is there a way to see why its not scaling? Maybe a log?

6

If it’s not scaling down, then there’s something hitting your service to keep it alive. Usually it takes 7-10 minutes of idle before the fly proxy shuts down the machine.

7

Is there something I need to change to stop sending traffic to machines when not needed?

8

Isn’t that statement contradicting? If there’s traffic, then the machine is needed.

9

Right now I have 10 machines and they are all actively taking requests. From the dashboard, they are all around “700 App Concurrency”. Shouldn’t they autoscale down until the soft_limit is hit which is 8,000? Technically one server should be able to handle this in my scenario.

10

There was a recent bug w/ scaling, so maybe you’re affected by that. I’d wait a bit and see if that’s related. If not send an email to support.

1 Like
11

Is this intentional? Try changing it to requests.

12

Yes I believe that is correct because I am using the proxy handler.

handlers = ["proxy_proto"]
13

Was this bug shared? Where can I find more information?

14

but it’s the opposite problem you’re having.

15

Try changing type to requests. Perhaps you have a client connection hanging onto each machine preventing it to autostop.

16

Pretty sure this would break my application since it expects proxy_proto

17

I mean that. I don’t use proxy_proto so I don’t know if it’s requires type: connections. I look at the fly docs and it didn’t mention any requirements for that.

18

@dangra could my issue be related to the fix Fly just released around autoscaling?

19

@timl1 it could be related, the mentioned bugfix prevented changes to service definitions to propagate to fly-proxy. Your experiments changing soft_limt, suspension types and so on may not have been in effect unless you also recreated machines.

20

Thanks! So to test soft_limit changes I need to recreate the machines? Is there an easy way to do that or scale down and then back up?