Is it possible to auto_start auto_stop based on internal traffic?

, , ,
1

We have a service with GPU that we would like to stop when no requests are coming internally.

We use the service.internal address to make requests to this service, is stopping automatically but not restarted again.


[mounts]
source = 'gpu_data'
destination = '/data'

[[services]]
internal_port = 8000
protocol = "tcp"
auto_stop_machines = true
auto_start_machines = true
min_machines_running = 0

[[services.ports]]
handlers = ["http"]
port = 80

[[services.ports]]
port = 443
handlers = ["tls", "http"]

[services.concurrency]
hard_limit = 5000
soft_limit = 4000
type = "connections"

[[vm]]
size = 'l40s'
memory = '32gb'
cpu_kind = 'performance'
cpus = 8

[[services.tcp_checks]]
grace_period = "1s"
interval = "15s"
restart_limit = 0
timeout = "2s"
2

Hi… You’ll need to use Flycast, since auto-start is a function of the Fly Proxy. (In contrast, the .internal addresses carry direct machine ↔ machine traffic.)

Try http://service.flycast:80 after fly ips allocate-v6 --private.

Caveat: it’s port 80 in this new context, instead port 8000—and there is no SSL.

1 Like
3

Added gpu, machines

4

Gonna try, thanks! Really appreciated for the help, has a lot of sense. I didn’t know that .internal was not going through the proxy.

Indeed, that was the fix!! Thanks you so much

1 Like
5

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.