tl;dr: $0.15/GB of RootFS for a machine stopped for 30 days.
Back when we announced machines in May 2022 we were still figuring some billing for it. We mentioned we would charge for running machines the same as we did for Nomad VMs but for stopped machines we asked for community feedback.
Fast forward almost 2 years later (I can’t write a billing post without mentioning how bad we are at charging customers, ikr) and we are preparing things for finally doing it and I’m here to answer all your questions too. We are not activating any billing job and we are giving you 3 months heads up too. And this time we will nag everyone via email a lot too.
We are going to charge $0.15 per GB/month of used RootFS while your VM is stopped. And no this is not going to kill our free allowances (more below).
So what exactly are we charging you for?
Machines are fast booting VMs. That means we have to keep your built RootFS around so when the machine starts it already has all your NodeJS/Elixir/Erlang/whatever runtime is already installed.
Take for example the machine I just showed above: if I stop this machine, our servers will have a 905MB volume sitting there until the machine starts again, which could be in an hour or in months. We do have folks with stopped machines for months and those cost us.
What the heck is RootFS?
Whenever you spawn a machine we always give you a secret-not-so-secret volume called RootFS that is mounted at, you guessed it, your root directory. Here’s an example:
[I] ➜ fly ssh console -a fly-replay-app
root@6e82554b262e87:/app# df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 97M 0 97M 0% /dev
/dev/vda 7.8G 191M 7.2G 3% /. <------ LOOK HERE
shm 109M 0 109M 0% /dev/shm
tmpfs 109M 0 109M 0% /sys/fs/cgroup
Its usually called /dev/vda. This RootFS is 8GB long but only used 191MB. We are going to charge only for the used RootFS for stopped machines.
When you build your machines you generate an OCI image that is needed to be used in your deploys. When your machine is first created we unpack this image into a real File System on your machine.
How is this related to my actual machines anyways? RootFS is ephemeral meaning it resets after the machine is stopped and it goes back to the starting usage. Take from example the image above, my RootFS usage showd on the dashboard is 905MB meaning if I stop my machine it will need from the host server 905MB of storage as long as this machine is stopped. It does not matter if at runtime I download many GB of cache files or if I remove many MBs of files, the RootFS will still be 905MB as long as the machine is stopped.
How do I calculate this charge?
- Usage is only calculated for stopped machines, if your machine is running 24/7 you don’t have to calculate anything, RootFS is already included on the price you already pay for running machines regardless of how much you use it.
- Usage is calculated on the amount of RootFS needed when the machine is stopped. RootFS is ephemeral meaning anything you modify after the machine starts running will not be charged.
- Each used GB costs $0.15 if it runs for 30 days, we prorate per hour.
Here’s a few examples:
My machine was stopped for 30 days this month and the rootfs usage is 100MB
That’s an easy one: $0.15 (1GB/month charge) * 0.10 (only 100MB) = $0.01
My machine spent 10 days stopped then ran for 20 days this month. It had 2GB of used rootfs.
$0.30 (2GB/month charge) * 0.33 (only 1/3 of the month) =~ $0.099
Is this the end of Fly.io free allowances?
No. Our free allowances let you run ‘Up to 3 shared-cpu-1x 256mb VMs’ 24/7. We also have been discounting up to $5.00 of usage on invoices so that means if you have 3 stopped machines for the entire month each having 1GB of RootFS usage that is $0.45 so far from breaking your free allowances.
How to verify how much I’m going to get charged?
We also shipped a preview in your invoice page that tells you how much dedicated usage you have so far in the current month prorated by hour. These will not be added to your invoice right now but by April 25th they will be added for May’s invoice.
We started collecting those preview charges in January 22th. Another way to see your usage is to go to our usage dashboard at https://fly.io/dashboard/personal/usage/stopped-machine.
How to verify I’m not going to be charged on the free allowances?
You can let your 3 shared-cpu-1x 256mb VMs running for the entire month or another way is to check that your stopped machines won’t add up to several GBs of RootFS usage together. You’d need more than 33GB of RootFS usage assuming all your machines were stopped for the entire month to exceed the $5.00 usage discount, for example.
How to pay less?
Another pattern y’all can find in my billing posts is “how to pay our company less money” so here we go.
It all depends on what you install on your OCI image. RootFS essentially is what your OCI image has after its packed + a few Containerd tweaks to make the file system usable (that’s why the value you see from docker build . might differ from our dashboard values for RootFS).
If your image install less stuff it means you need less base RootFS which means we are going to charge you even less. Believe it or not we do want you to have smaller images because that means faster builds too! Take @rubys work on dockerfile-node for example: we use multi-stage builds to make your images weight even less by not storing all the things needed just to compile your app like babel compiler or whatever.
So try multi-stage builds, try alpine if you want, it might save you some money and make your deploys even faster although you might want try to calculate if this is going to save you cents or fractions of cents. Unless you’re one of the customers racking several stopped machines you don’t really need to overthink this.
What’s next?
We are going to send emails. A lot of them. Plus we still need to show RootFS usage to our CLI, currently only our UI shows it. We need to update our pricing docs too!
We still want to hear it from you though. Let us know if you have any questions about all of this.
