So I just added a secret to a very, very simple Rails app we run on Fly.io. In the activity log, I see that that is followed up by a new release.
Due to this release, my app was not available for over 10 seconds. What is the logic behind completely taking the app offline when a secret (ENV var) is added?
I believe Fly defaults to a new release when secrets are changed. Arguably that shouldn’t include a secret being added as that could not affect existing code (which would not know about that secret even being there) but it would (potentially) affect a running app if a secret is updated or deleted.
As regards taking the app offline, I think that is an unintended consequence of any deploy/release (currently), rather than due to secrets specifically. The current solution is to have an app with at least three instances. See e.g:
Thanks for the insightful answer @greg!
This is exactly what I was looking for, however I didn’t fully understand the latter part of your answer yet. I do have to say my collegue @qqwy has explained the same thing before, I’m just a little slow in understanding I guess.
This seems like a very elegant solution by Fly by the way. We’re deploying a couple of our internal services on Fly, hoping that we can migrate with our main app. Very happy so far!
You’re welcome.
Kurt posted today they are close to fixing it 
It’s because of how Fly’s innards work. The “3+ VMs” solution just masks it for now. There is some more detail from @kurt here on the cause:
The secrets require all the VMs to restart because they’re exposed as environment variables. At the moment, there’s no way to add a secret without it immediately being applied in a release. Staging secrets is something we know we need to do, and it should pretty quick after some very not-quick architectural changes we’re making.
Just so you know, you shouldn’t see downtime when you update secrets any more.
