VM size planning

We’re exploring GPUs! They’re a hard problem because GPUs are meant for multinenancy, so you have to figure out how to allocate an entire GPU to a VM. That also means they’re expensive.

Right now, we have two classes of vms. shared-cpu and dedicated-cpu. We’re planning to release:

  • Larger shared CPU VMs, probably shared-cpu-2x, 4x and 8x. This will you give you shared access to up to 8 CPUs. They might be pretty good for things with bursty CPU.
  • More dedicated CPUs. Right now we only go to 8. We could theoretically go to 24.

We’ve also been considering more permanent “reservations” for dedicated CPU VMs.

What kinds of things could you all use?

It could be really nice to have a CPU option optimized for background/offline jobs, where it’s de-prioritized to only fill otherwise-unused CPU capacity. There must be some kind of cgroup magic that can be done to achieve that. Just to be clear, this would only make sense if that CPU option were also cheaper.

If you could “innovate” even further and make that CPU class only charge for core-seconds actually consumed, that would be amazing.

  • For CI, having a dedicated with high CPU would improve speed
  • 4GB memory shared cpu would be great for a few of our services that aren’t used often but load large models.
  • For more info, the GPU is really not a big priority, so don’t count it as a strong vote from me