Just implemented a Node v2 app with multiple machines to encode audio with scale to zero. Couldn’t have been easier.
1. Sleep your app after some inactivity
This will be different depending on what you’re using. In Node it can be as simple as doing this, and updating the timeout on every new request.
let timeout;
export function resetCountdownSleep () {
clearTimeout(timeout);
timeout = setTimeout(() => {
process.exit();
}, 60000);
}
In this case, the server will sleep after 60 seconds of inactivity.
Remember to start the timeout when the app starts too, because otherwise it will remain awake after you deploy.
2. Set the restart policy
You need to set the restart policy to no
, otherwise the machine might wake up automatically after going to sleep which would defeat the purpose of scale to zero
fly machines update <machine-id> --restart no
3. Set the concurrency
So now, Fly will start waking up the machines as requests start coming but you might wanna change your concurrency settings in the fly.toml
settings. If you have multiple machines, Fly will not wake up a new machine until the soft limit of the concurrency has been reached which by default is 25 I think.
In my case, I’m doing a CPU heavy job so generally I don’t want a machine processing more than one request at a time.
[services.concurrency]
type = "connections"
hard_limit = 3
soft_limit = 1
I have enough machines to handle enough audio encoding jobs. But in case of a huge peak comes up I set the hard concurrency to three.
And thassit!