Fly Apps have been a great way to isolate your customers from each other by ensuring they each have their own sandbox while also avoiding co-mingling of data. However, this approach didn’t work well with our metrics-based autoscaler, fly-autoscaler, since it could only scale a single app. For example, if you had 100 apps then you’d need 100 fly-autoscaler instances. What a pain!
But today we released fly-autoscaler v0.3.0 which adds the ability to manage a whole fleet of applications with a single autoscaler instance!
You can find an example of multi-application scaling in the fly-autoscaler-multiapp-example repository.
How it works
We’ve revamped the autoscaler to start a pool of reconcilers instead of using a single one and those reconcilers will work through a list of applications that you provide to ensure they at the optimal scaling state.
Configuring the application list
Most of the configuration works the same as single-app autoscaling but there are a few tweaks. First, you’ll need to specify the organization that your apps belong to and you’ll need to use a wildcard (*) to match against the set of apps you want to scale. Multi-application scaling only works with a single organization.
FAS_ORG="my-org"
FAS_APP_NAME="my-app-*"
The autoscaler refreshes the list of applications in an org from our API on an interval (every minute by default) and will match against those apps using the expression above. For example, this will match "my-app-1" and "my-app-2" but not "some-other-app".
Use an organization-wide token
Since we’ll be scaling multiple applications, we’ll need to switch from single-application auth token to an organization-wide auth token. You can create one with flyctl:
fly tokens create org -o my-org
And then set this token in your autoscaler application:
fly secrets set FAS_API_TOKEN="FlyV1 ..."
Using the application name in the metrics query
The next step is to update your Prometheus query to utilize the application name. This query will be executed once for each application as it’s being reconciled and scaled and the autoscaler will substitute any instance of $APP_NAME or ${APP_NAME} with the name of the currently scaled app.
For example, if you’re tracking the number of connections on each app and you want to scale based on that value, you could use a Prometheus query like this:
FAS_PROMETHEUS_METRIC_NAME = "connection_count"
FAS_PROMETHEUS_QUERY = "sum(connection_count{app='$APP_NAME'})"
Upcoming autoscaler work
We have some more metrics sources we’ll be adding to the autoscaler soon such as a Kafka souce & a NATS source. We’ll also be putting together some official docs this week to make it easier for folks to get up and running.
Please let me know if you have any questions or feature requests!