Auto extend volume size after a certain threshold!

tl;dr: We just shipped an API feature that lets you auto extend volume sizes when usage hits X% (you can configure that).

Good morning, folks. Im glad to announce something that I’ve been working for a while now. We just shipped a few hours ago auto volume extensions. The gist of it is that you need to add this to your machine config mounts section.

... rest
    "mounts": [
      {
        "volume": "vol_zred08l8yjx96yl4",
        "path": "/data",
        // New stuff below!
        "extend_threshold_percent": 80, // Auto extend if uses 80% storage
        "add_size_gb": 10, // Auto extensions add 10GB
        "size_gb_limit": 50 // Do not go over 50GB total
      }
    ],

All of those fields are entirely optional so no need to change your API usage. Here’s everything you need to know about this.

What’s the minimal config that I need to do?

To make auto extensions work you need at least an extend_threshold_percent and add_size_gb under your mount. size_gb_limit is entirely optional and works so you can have predictable billing on your machine.

How much does this costs?

This feature is opt-in so we are not making your volumes magically cost more.

We haven’t changed anything in the way we bill volume usage either. Combining add_size_gb and size_gb_limit you can ensure that the amount you’re at most getting charged for is predictable.

How to disable this?

Simply do not send those fields or send values as 0.

Recommendations

  • Try to use a reasonably large extend_threshold_percent such as 75 or more so you don’t get a loop of auto extensions.
  • Find a good add_size_gb that fits your scenario. Feel free to even use 1GB per extension if you want, just make sure you understand how fast your disk is filled so if you feel like 1GB might not be good enough for whatever you do with volumes consider bumping this value.
  • Be careful about temporary data. Take Postgres for example, it can build WAL files that will temporarily increase your volume usage so take those in consideration as our auto extension will trigger as soon as your threshold hits, it’s not aware of temporary files.

Next steps

This is very fresh so we, we just update api.machines.dev to add those fields but we still need to do some docs around it. FlyCTL is next in our radar so you will be able to use it by just adding things to your fly.toml

But we were so eager to show you this feature so we could gather your feedback as soon as possible. Is this useful for you? Do you feel like this could have been better? Let us know!

10 Likes

Awesome!

Does the 500GB volume limit still apply?

1 Like

I really want to be able to configure this from fly.toml very soon!

1 Like

It does apply. For bigger volumes we offer an alternative too:

1 Like

When will this be available for fly.toml?

1 Like

Im currently working on this. Spoilers: [wip] auto volume extension from flyctl by lubien · Pull Request #3021 · superfly/flyctl · GitHub

1 Like

Oh, nice! I just subscribed the thread!

1 Like

It’s done!

2 Likes

Of course I enabled it an hour after it was released! :laughing:

2 Likes