That’s a good question, actually… Qualitatively, it’s too slow to be done interactively. (Not with much reliability, in my view.)
For example, if this was a video subtitling service, with the user uploading a file and then pressing Go, and getting reassuring “currently processing this scene” thumbnails as it progresses, then when you do receive the POST request at your end… it’s probably already too late to be creating the corresponding worker Machine now.
The “instantaneous demand” would be the number of workers strictly needed for the users who have already pressed Go, basically. More sophisticated capacity management would take into account signals about the shape of the near future, like historical statistics for this time of day, how many users have the submission page open but haven’t really done anything with it yet, etc., but even simplistic measures like “just keep some spares around” seem to yield gigantic improvements for people.
Moreover, the Machines platform concept itself revolves around pre-creation (so to speak), with auto-start (and now auto-unsuspend) intended for immediate demand response. (Waking up from suspend in particular is very fast. I wonder sometimes whether they’re singing the praises of that insufficiently, actually, now that it’s exited(?) its lengthy and mildly murky experimental phase.)
Fly.io has mentioned in the past that creation time depends a lot on the exact structure of your Docker image, and they’ve also been doing extensive work themselves behind the scenes on things like region-optimized image operations, so latencies have been getting better. (And, indeed, there are far fewer complaints lately in the forum about slow Machine creation.)
Still, I don’t think “ton of variance” itself is going out on a limb,
. I see easily 10× discrepancy (on the same image) with my own Machines, and those are at the smaller end of the size spectrum.
I wish I also had quantitative feedback, actual numbers, to give you at this point, but I mainly only use Fly.io for ad hoc experiments myself. In addition to doing your own measurements, it might be fruitful to post a new top-level topic asking particularly about people’s recent Machine-creation statistics. There are some avid data hounds in the forum here, so odds are good that you’ll get some…