Implementing scale to zero is super easy!

1

Just implemented a Node v2 app with multiple machines to encode audio with scale to zero. Couldn’t have been easier.

1. Sleep your app after some inactivity

This will be different depending on what you’re using. In Node it can be as simple as doing this, and updating the timeout on every new request.

let timeout;

export function resetCountdownSleep () {
	clearTimeout(timeout);

	timeout = setTimeout(() => {
		process.exit();
	}, 60000);
}

In this case, the server will sleep after 60 seconds of inactivity.

Remember to start the timeout when the app starts too, because otherwise it will remain awake after you deploy.

2. Set the restart policy

You need to set the restart policy to no, otherwise the machine might wake up automatically after going to sleep which would defeat the purpose of scale to zero :slight_smile:

fly machines update <machine-id> --restart no

3. Set the concurrency

So now, Fly will start waking up the machines as requests start coming but you might wanna change your concurrency settings in the fly.toml settings. If you have multiple machines, Fly will not wake up a new machine until the soft limit of the concurrency has been reached which by default is 25 I think.

In my case, I’m doing a CPU heavy job so generally I don’t want a machine processing more than one request at a time.

  [services.concurrency]
    type = "connections"
    hard_limit = 3
    soft_limit = 1

I have enough machines to handle enough audio encoding jobs. But in case of a huge peak comes up I set the hard concurrency to three.

And thassit!

11 Likes
2

The default restart policy is always . So If you don’t change it to on-failure or no the VM will always reboot itself.

I spent some time trying to figure it out a few days ago. :face_with_monocle:

2 Likes
3

Neat.

For a bursty service, say 1000+ reqs per second, clearing and setting timers discretely once in a while might be more desirable:

// untested 
const onemin = 60_000 // ms
let prev = Date.now();
let countdown = setTimeout(quit, onemin);

function quit(code = 0) {
   return process.exit(code);
}

function reset(ms = 0) {
	clearTimeout(countdown);
	countdown = setTimeout(quit, ms);
}

export function keepalive(ms = onemin) {
    const now = Date.now();
    const elapsed = (now - prev)|0;
    const dist = elapsed - ms;
    const recent = elapsed <= (ms*0.33);
    const old = dist >= (ms*2);

    if (!recent || old) {
        reset(ms);
        prev = now;
    }
}

We use a variant of the above for our Fly (tcp) service (code).

2 Likes
4

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.