Hi, I was trying to track down some mystery file usage today for my pg db and I found where it was.
TLDR: I think there’s a bug in the way the dashboard shows “Used” storage for a volume. The default settings on my db’s volume’s filesystem had a 5% reserved block ratio applied and shown as part of the usage in the dashboard, when that space is actually usable.
In other words, for my 400 GB volume, 20 GB were “used” right off the rip according to the dashboard.
Before (with all default settings – I hadn’t fiddled with this database or volume at all):
root@080e337f007748:/# df -h /data
Filesystem Size Used Avail Use% Mounted on
/dev/vdc 394G 32G 347G 9% /data
After applying tune2fs -m 0.5 /dev/vdc
(DO NOT RUN THIS COMMAND):
root@9185e256a99d78:/# df -h /data
Filesystem Size Used Avail Use% Mounted on
/dev/vdc 394G 32G 346G 9% /data
I put the disclaimer in because you need to unmount the filesystem to not corrupt it with that command, and it’s not necessary anyway. The command changes the filesystem’s reserved blocks ratio from the linux-default 5% to 0.5%. Reserved blocks are filesystem blocks that are reserved for root so the OS remains usable if space runs out. Since fly machines only use root, all of the reserved blocks are usable by an app anyway.
You’ll note the screenshots are on two different volumes. That’s because I created a separate PG app and volume based on a snapshot of the same database a couple hours apart to validate this.
I sent an email to fly support now with this info, so we’ll see what they say.