I am an MSc student currently writing his master’s thesis.
In the process I measured the execution time of Java workloads on different FaaS(-like) platforms from within the code. Though Fly generally compared favorably to other platforms, it completely surpassed my expectations on a simple CPU bound workload. Fly was able to finish a prime number calculation by unoptimized trial division in less than half the time AWS Lambda (or our own server) needed. Since the code and resource allocation were as similar as possible, I didn’t expect such a huge difference.
I tried to investigate the cause, but didn’t find a satisfying explanation, so I would be very thankful if I could get a short statement on this to include in my thesis.
The results you got are probably a combo of a few things. We run VMs on big, beefy boxes with AMD Epyc CPUs. Most of the hardware has 24 cores /48 threads, some has 32c / 64t. We provision up to 8 or 16 VMs per core, if they were all blasting the CPU you’d get worse performance.
Most VMs aren’t CPU bound, so your VM will most likely get scheduled on a hardware thread that’s close to idle.
That said, is there a minimum CPU resource guarantee in a worst case scenario (i.e. the other 15 VMs are blasting the CPU)?
We have an I/O bound task (nginx serving static files) and were thinking if a pool of shared-cpu-1x instances could work reliably for our case (sustained load of 300-500 requests per second).
Yeah we’re using cgroups + consistent fair scheduling. If all the VMs on a host are running prime number generators, a shared-cpu-1x vm will effectively get 1/15th of a physical hardware thread. This is a very theoretical case, though, we’re likely to shuffle VMs around long before that happens.
shared-cpu-1x servers will work great for nginx, though, we have a bunch of people doing exactly that.
