We’re looking to streamline our secrets mgmt. Currently our CI/CD pipeline handles deployment of the app using flyctl, but we have no means of guaranteeing an app and secrets are deployed together.
In previous setups, i’ve handled this via sealed secrets and config as code… where manifests reference the secrets for their deployments, but not sure how to solve w/ fly.
As a step toward keeping applications + env configuration in lock step, is there a way to “set” secrets in a “deploy” CLI cmd, allowing the caller to maintain the versioning logic but guarantee the app and secrets are deployed together or not at all?
As i understand it, running a deploy and a secrets set command separately will yield separate deployments for the code and the secrets. Setting secrets first without triggering a redeploy is still destructive to the previous secret values in case a subsequent deployment of new code fails.
Consider the scenario where [app_v0, secrets_v0] are both deployed. The user wants to deploy [app_v1, secrets_v1] together, because either combination of [app_v0, secrets_v1] or [app_v1, secrets_v0] would NOT be a viable deployment.
How can [app_v1, secrets_v1] be deployed TOGETHER, with rollback to [app_v0, secrets_v0] in the event of a deployment failure.
Perhaps there is a way but personally I don’t know of one.
I had two ideas which aren’t exactly what you are after but perhaps get a bit closer.
Use fly deploy --env NAME=VALUE. That reads like you can set run-time values at the same time as the deploy but they wouldn’t be encrypted and so wouldn’t be secret. If your values have to be kept secret, that wouldn’t be the solution.
First run fly deploy --build-only. That would build e.g [app_v1] but not deploy it. At this point you would still have [app_v0, secrets_v0] deployed. But have [app_v1, secrets_v0] ready. Then you’d run fly secrets set NAME=VALUE. Running that command updates the secrets and triggers a new release (as new secrets are only pulled in when a VM starts). In theory … that should release the latest built image. Which would be … app_v1. And so you would then get [app_v1, secrets_v1] deployed.
I think you want the --stage flag when setting secrets (docs)
The --stage flag:
--stage Set secrets but skip deployment for machine apps
This is for “machine apps” which are our “appsv2”. If you run fly status on your app, you’ll see if it’s “nomad” based or “machines” based. You want machines.
So, in your pipeline, you could do something like this pseudo code:
foreach(secrets as key, value):
fly secrets set "key=value" --stage
endforeach
fly deploy
API Calls instead?
The other thing you can do is call the equivalent GraphQL API call directly instead of using the CLI (this is the same as the --stage flag, the secret gets set but no deployment is kicked off).
the --stage flag is interesting. Any documentation on how it behaves? Will restarts of the existing deployment still use the older secrets (pre staging the new secrets).
CI/CD scenario, w/ existing deployment of [app_v0, secrets_v0]