Using fly.io as an alternative to AWS Lambda

Questions / Help
1

I have an application that is currently based on AWS Lambda, and I’d like to move it to Fly.

The application expects Lambda-like semantics, i.e. only one request can be processed in any given container, and given the CPU intensive nature of the task, it is preferable to keep it that way. I expect loads of 40 concurrent requests, with occasional spikes to 500 concurrent requests.

While fly.io now supports scale to zero and auto_start/stop_machines, how would I go about configuring fly so that only one request is handled at any given time?

(I had asked this question a long time ago, but now that v2 apps have been launched, I’m guessing the answer is different.)

2

Set both the soft_limit and hard_limit in your service definition to 1: Load Balancing · Fly Docs

Keep in mind that apps v2 does not autoscale. Your maximum amount of concurrent requests will be the amount of machines you have created.

3 Likes
3

Thanks for replying! I find the last part confusing though, and thus it’d help me a ton if you could clarify these questions:

  1. The fly.toml reference mentions mentions under “auto_start_machines” that it controls “[w]hether to automatically start an application’s machines when a new request is made to the application”, does this control only the waking up of the machine when a request is received? The way this is worded sounds like this is the setting that controls autoscaling.

  2. Would Scale the Number of Machines · Fly Docs be the way I set the number of machines? Is there a way I can set it in fly.toml?

  3. The Fly documentation mentions I’m not billed for stopped machines, which means if, say, if I scaled my app to 500 machines but used no more than a day’s worth of compute time in total across all machines, I’d be only billed for that day’s worth of compute time and no more?

4

Hi @Biswas

Fly doesn’t currently autoscale machines in the traditional sense that it creates and destroys machines based on load.

Instead it has autostart/autostop to start and stop machines based on load. You still need to create all the machines beforehand and the number of machines you create will be your maximum scale.

You can scale machines using the fly scale count command, but also by machine cloning (which is a slightly more complicated concept and is not necessary).

At the moment, for stopped machines you are billed for the storage that stopped machines use (though at the moment I think fly are providing it for free until some point in the future). You are not billed for cpu, memory, or bandwidth for stopped machines.

2 Likes
5

Thanks for your help again, though I had a question about the quoted part.

When we provision a fly container, is it completely read only (including /tmp) and the only way to obtain writable storage is by provisioning a volume?

Because volume pricing is the only storage pricing I find being mentioned.

6

No, the VMs are not read only.

Each VM receives a roughly 10GB ephemeral volume (meaning it gets deleted when the VM shuts down) which all the files from the docker image is loaded into (as well as all the extra stuff the fly needs to run a VM).

This ephemeral volume is not priced separately and is included in the pricing for the VM.

This means you can write to /tmp or any other directory, so you could use it for temporarily storing file uploads, storing temporary logs, storing working data during image/video manipulation, etc.

4 Likes
7

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.