Since memory is one of those things most people would rather not care or worry about and Fly has a deep sentiment on “scaling where you need it” - Why not auto scale vertically as well?
The idea is that a VM would have a similar concept as an app has a min/max setting on VMs in its pool, an app would also have a setting for a min/max VM size, and if memory usage starts to reach +X% for Y time, a second VM would be spun up as a sibling with the next size up, once up and running, the smaller VM would then be removed from the pool. Same would go for when a VM is -X% for Y time it would then scale back down.
This is on the roadmap! The problem right now is we can’t dynamically resize VMs, and we prefer to not restart things if we can help it. We’re working on VMs with permanent storage that we think this will work really well for, someday!
+1 for this feature request. Especially now that Volumes exist. For example, if most of the time my app only has 0 to a few concurrent connections, it could be a single instance on the shared-cpu-1x size. If there are peak traffic periods with 100 concurrent connections, it would be great if it could automatically scale up to a single instance on dedicated-cpu-8x before spinning up additional instances. That way, I’d only need a single volume (or maybe 2, for the time that a new size is spinning up) rather than dozens or more for the equivalent compute power spread horizontally across that many small instances.