fly-autoscaler v0.2.2: Now with Temporal Cloud support!

Fresh Produce
,
1

Our metrics-based autoscaler, fly-autoscaler, has now added Temporal support! Temporal is an open-source project based on Uber Cadence which provides “durable execution”. Durable execution is a bit like traditional async jobs but it allows you to model more complex workflows and ensure they are safely and correctly executed.

How autoscaling works with Temporal

We’ve added a new Temporal metric collector to fly-autoscaler which will periodically check for the total number of workflows in a “running” state. By default, it will check every 15 seconds.

The autoscaler can be setup to create or destroy machines for a worker application based on the current workflow count. For example, if your my-worker-app can handle 10 workflows at a time, you can configure the autoscaler with the following expression:

FAS_CREATED_MACHINE_COUNT="workflow_count / 10"

The autoscaler creates machines in an application by cloning them so it will never scale your app below 1 machine. If you want to ensure you don’t exceed a certain number of machines then you can use a min() expression to cap it:

FAS_CREATED_MACHINE_COUNT="min(50, workflow_count / 10)"

This will ensure that you never create more than 50 machines, regardless of how many workflows are executing.

Configuring fly-autoscaler for Temporal

We’ve provided a fly-autoscaler-temporal-example for reference but setting up the autoscaler just takes a few steps:

First, create a fly.toml for your scaler with the following build and env sections. Replace your Temporal address and namespace and your worker app name with your own.

[build]
  image = 'flyio/fly-autoscaler:0.2.2'

[env]
  FAS_APP_NAME = 'my-worker-app'
  FAS_CREATED_MACHINE_COUNT = 'workflow_count / 10'
  FAS_TEMPORAL_ADDRESS = 'mynamespace.lyeth.tmprl.cloud:7233'
  FAS_TEMPORAL_NAMESPACE = 'mynamespace.lyeth'
  FAS_TEMPORAL_METRIC_NAME = 'workflow_count'

Next, you’ll need to set secrets for your Temporal certificate & key data that you obtained while setting up your Temporal account. The best way to do this is with command substitution:

fly secrets set --stage FAS_TEMPORAL_CERT_DATA="$(<ca.pem)"

fly secrets set --stage FAS_TEMPORAL_KEY_DATA="$(<ca.key)"

Questions?

Let us know if you have any questions or issues setting this up. Please let us know if you have any suggestions for other features as well!

2 Likes