Persistent volume write performance

Hello,

I have an application that sometimes needs to write a 200 Mb text file to the machine’s attached volume. Until about the beginning of April, this process always took no more than 2 seconds. Now it takes at least 10 seconds. Sometimes more than 15 seconds.

Is this normal? Has anyone experienced the same drop in I/O performance?

Here is the output of the dd command from the laptop and from my fly.io machine:

Laptop

$ dd if=/dev/zero of=test bs=1M count=200 conv=sync
200+0 records in
200+0 records out
209715200 bytes (210 MB, 200 MiB) copied, 0,0671866 s, 3,1 GB/s

Machine

$ dd if=/dev/zero of=test bs=1M count=200 conv=sync
200+0 records in
200+0 records out
209715200 bytes (200.0MB) copied, 10.241086 seconds, 19.5MB/s

Hm… I do see that same slowness on the low-budget shared vCPU Machines, :raggedy_rabbit:—but not with Fly.io’s specifically performance set…

root@e82123abcdef98:/data# echo "$FLY_REGION"
ewr
root@e82123abcdef98:/data# dd if=/dev/zero of=test bs=1M count=200 conv=sync
200+0 records in
200+0 records out
209715200 bytes (210 MB, 200 MiB) copied, 0.425327 s, 493 MB/s

The existence of such a discrepancy isn’t super-surprising, since the shared class doesn’t consist of the fastest hares in the warren, overall. Even so, it’s possible that this is an unintended interaction of CPU throttling with virtualized I/O. (I seem to recall Fly.io mentioning that part of Firecracker’s own operations, on the host rather than the guest side, were also caught up in the throttle.) I’d suggest taking a look at the Grafana panel mentioned in that earlier thread.

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.