I have an application that is currently based on AWS Lambda, and I’d like to move it to Fly.
The application expects Lambda-like semantics, i.e. only one request can be processed in any given container, and given the CPU intensive nature of the task, it is preferable to keep it that way. I expect loads of 40 concurrent requests, with occasional spikes to 500 concurrent requests.
While fly.io now supports scale to zero and auto_start/stop_machines, how would I go about configuring fly so that only one request is handled at any given time?
Thanks for replying! I find the last part confusing though, and thus it’d help me a ton if you could clarify these questions:
The fly.toml reference mentions mentions under “auto_start_machines” that it controls “[w]hether to automatically start an application’s machines when a new request is made to the application”, does this control only the waking up of the machine when a request is received? The way this is worded sounds like this is the setting that controls autoscaling.
The Fly documentation mentions I’m not billed for stopped machines, which means if, say, if I scaled my app to 500 machines but used no more than a day’s worth of compute time in total across all machines, I’d be only billed for that day’s worth of compute time and no more?
Fly doesn’t currently autoscale machines in the traditional sense that it creates and destroys machines based on load.
Instead it has autostart/autostop to start and stop machines based on load. You still need to create all the machines beforehand and the number of machines you create will be your maximum scale.
You can scale machines using the fly scale count command, but also by machine cloning (which is a slightly more complicated concept and is not necessary).
At the moment, for stopped machines you are billed for the storage that stopped machines use (though at the moment I think fly are providing it for free until some point in the future). You are not billed for cpu, memory, or bandwidth for stopped machines.
Each VM receives a roughly 10GB ephemeral volume (meaning it gets deleted when the VM shuts down) which all the files from the docker image is loaded into (as well as all the extra stuff the fly needs to run a VM).
This ephemeral volume is not priced separately and is included in the pricing for the VM.
This means you can write to /tmp or any other directory, so you could use it for temporarily storing file uploads, storing temporary logs, storing working data during image/video manipulation, etc.