Since flyctl v0.0.522, it is possible to have different mounts per process group on V2 applications. This means your web app and your background jobs can have attached volumes from the same pool, different pools, or none at all. This was possible with Nomad but was missing for V2 apps until now.
This is useful when an app is composed of different process groups (like web, background jobs, stateless components) that have different volume requirements (size, count, encryption, …).
For example: let’s say we want to build a video converter app that keeps metadata in sqlite (backed by litefs cluster). The app has a background job for converting uploaded videos that requires 50GB of disk space. Once converted, the videos are uploaded to S3 and removed to free up space. As a plus, there is a process that consumes messages from the task queue and sends notifications when conversion are done, and it doesn’t need any volume to work.
This is the example
[processes] app = "/start webserver" video = "/start video-converter" notifier = "/start notifier" [http_service] internal_port = 8080 force_https = true processes = ["app"] [[mounts]] source = "litefs" destination = "/.litefs" processes = ["app"] [[mounts]] source = "stage" destination = "/stage" processes = ["video"]
And here how to launch this app:
$ fly launch --force-machines -o personal --region iad --copy-config --now --name mounts-per-group ... Creating 1GB volume 'litefs' for process group 'app'. See `fly vol extend` to increase its size Creating 1GB volume 'stage' for process group 'video'. See `fly vol extend` to increase its size Process groups have changed. This will: * create 1 "notifier" machine * create 1 "video" machine * create 1 "app" machine No machines in group 'app', launching one new machine Machine 148ed590a04258 [app] update finished: success No machines in group 'notifier', launching one new machine Machine 148edd77f75548 [notifier] update finished: success No machines in group 'video', launching one new machine Machine e2865649c53328 [video] update finished: success Finished launching new machines Updating existing machines in 'mounts-per-group' with rolling strategy Finished deploying
I removed the uninteresting parts from above log.
fly launch is creating 2 volumes (one per group) and launching 3 machines, two of which have mounts.
$ fly status App Name = mounts-per-group Owner = personal Hostname = mounts-per-group.fly.dev Image = mounts-per-group:deployment-01GY8HXKB7A6FG17JF8W0AYPA3 Platform = machines Machines ID PROCESS VERSION REGION STATE HEALTH CHECKS LAST UPDATED 148ed590a04258 app 1 iad started 2023-04-17T21:29:28Z 148edd77f75548 notifier 1 iad started 2023-04-17T21:29:32Z e2865649c53328 video 1 iad started 2023-04-17T21:29:47Z p$ fly vol list ID STATE NAME SIZE REGION ZONE ENCRYPTED ATTACHED VM CREATED AT vol_ez1nvxk6zm8rmxl7 created stage 1GB iad 7806 true e2865649c53328 2 minutes ago vol_3q80vd3qnekrgzy6 created litefs 1GB iad 7806 true 148ed590a04258 2 minutes ago
Note: the only thing left is extending the volume to 50GB. I will leave that as a reader’s exercise
Anything missing? something you would like to see?
let me know!