We run Zelly (org : zelly), a consumer AI assistant. Each customer gets a
dedicated app (1x shared-cpu-1x 1024MB machine + 1GB volume) in CDG, plus a few
shared service apps. We’re currently at a couple dozen apps and planning to grow
to 500+ machines in CDG by end of 2026, provisioned progressively via the
Machines API.
A few questions before we scale:
Are there org-level limits (apps, machines, volumes) we should ask to raise
ahead of time?
What are the practical rate limits on the Machines API for provisioning
bursts (we may create up to ~20-50 machines/hour during signup spikes)?
Our fleet reconciler polls each machine (GET /v1/apps/{app}/machines/{id})
every 5 minutes, throttled at 5 req/s or less. At 500 machines that’s a
sustained ~1.7 req/s average. Is that pattern fine at this scale, or is
there a recommended bulk/list approach we should switch to?
Any capacity recommendations for CDG specifically (shared-cpu-1x + 1GB
volumes), should we spread across another region as a fallback?
Longer term our roadmap contemplates 5-10k+ machines, at what scale should
that become a conversation with your team rather than forum posts?
We just want to avoid hitting a wall mid-growth. Thanks!
Thanks for posting about this, I’m happy to address the questions!
There are limits on the total amount of machines you can provision. This helps aid capacity planning and ensures that a runaway script doesn’t provision thousands of machines before someone notices. This limit was set to the default of 100 on your account, and I’ve proactively raised it to 1,000. From the dashboard, if you click into an app, you should be able to see the machine limit as well as how many machines you’re using. When you need it raised in the future, you can write in to billing@fly.io and we’ll get it taken care of for you.
We document the API limits here: https://fly.io/docs/machines/api/working-with-machines-api/#rate-limits So for creating a machine you’re looking at 1 request per second as a baseline with short bursts to 3 requests per second. It sounds like you’ll fit within these limits well, but you should be prepared to handle errors from the API and then retry (ideally with a backoff like 1 second, 2 second, 4 second).
This should also be okay, Machine Info requests are rate limited to 5 per second, as long as you’re throttling them I don’t believe you’ll run into an issue. One optimization you can make here, if you have 2 or more machines in an app, would be to leave the ID off at the end. You should get results for all of the machines in that app (https://fly.io/docs/machines/api/machines-resource/#list-machines).
We will generally recommend you spread machines across multiple regions. You can check the current capacity for the type of machine you described by running fly machine place --region cdg --vm-size shared-cpu-1x --volume-size 1000.
I would recommend any time you’re planning to scale up more than 500-1000 machines and haven’t talked to us, reach out to billing@fly.io. We can make sure everything is handled well for you and get the machine limits adjusted.