Secrets not getting injected into newly deployed app

[yakkom]

Hey there! Unsure what the right place for this is but it seems like secrets are not being injected properly into my recently launched app.

I’ve experienced this a couple of times now. My secrets weren’t coming through, then they did, and now I’m trying to update a secret and it’s not coming through. When I SSH into a machine, the secret’s not there, even though it’s been set on the dashboard and multiple deploys happened after it.

[yakkom]

scaling to 0 and back up seems to do the trick

[mayailurus]

fly secrets sync would be less drastic, but glad you got something working!

https://community.fly.io/t/flyctl-secrets-sync/26044

You can read the original explanation of why there’s an extra step in the following announcement thread:

@Tim_Newsham

The secrets versions are now tracked in the flyctl config file. There’s a caveat here – if you manage secrets from flyctl on one machine and then perform a deploy from flyctl on another machine its possible that the secrets will not be up to date. This inconsistency can last for a short while until secrets are propagated and secrets in caches are aged out.

(The Fly.io dashboard would generally count as a second machine, in this context.)

[koz]

Just had the same issue with the Dashboard not propagating secrets to my containers. Even using the “Deploy Secrets” button didn’t work. Eventually I got it working with fly secrets import on the command line.

Not really a great experience to update my secrets re-deploy and have my app crashing with old secret values for an indeterminate amount of time.

[arnodirlam]

Same here. Yesterday, I updated an existing secret in the Fly Dashboard, then deployed the secrets. The env var on the new instance still had the old value though. Also deployed a new version with some changes, still the new value wasn’t picked up.

Eventually, using fly secrets set locally worked.

[yakkom]

Thanks for the response with the clarification! But yeah it’s a bit of a weird experience that the secrets dashboard has a “Deploy Secrets” button that does not effectively deploy secrets. Bit of an expectations mismatch there.

[aezell_fly]

I understand how this would be frustrating to click a button that says it’s going to do exactly what you want and then it doesn’t happen.

I just released a small change to this Deploy Secrets function that should force a secrets sync before the secrets deploy when using the web UI. This is similar to how you might use fly secrets sync and then fly secrets deploy from the CLI.

Do let us know if you continue to see secrets that aren’t deploying from the Dashboard correctly.