Setting secrets before the first deployment does nothing

1

If you use fly secrets set before the first deployment the secrets won’t be set.

The cli says Secrets are staged for the first deployment but after i tried to deploy secrets were not added.

2

Strange. They are set (or at least they were). Since I’ve done that, staging secrets and then deploying. So unless a recent bug has broken that behaviour, they should be there …

Can you confirm how you are accessing them? They should be environment variables so e.g for Node process.env.NAME. The other possibility is with a type error/conversion so e.g a number being converted to a string, and so not being correctly passed to the application. But that’s a total guess!

3

I have the same problem. I’m trying to fetch an env var in my config.exs using

  dsn: System.fetch_env!("SENTRY_DSN"),

and I set it for fly like so :point_down:

fly secrets set SENTRY_DSN=my_sentry_dsn

but then after I run fly deploy I get

-----> Installing Hex
** (ArgumentError) could not fetch environment variable "SENTRY_DSN" because it is not set
    (elixir 1.13.3) lib/system.ex:698: System.fetch_env!/1
    (stdlib 3.15) erl_eval.erl:685: :erl_eval.do_apply/6
    (stdlib 3.15) erl_eval.erl:893: :erl_eval.expr_list/6
    (stdlib 3.15) erl_eval.erl:237: :erl_eval.expr/5
    (stdlib 3.15) erl_eval.erl:229: :erl_eval.expr/5
    (stdlib 3.15) erl_eval.erl:893: :erl_eval.expr_list/6
    (stdlib 3.15) erl_eval.erl:408: :erl_eval.expr/5
-----> Installing rebar
** (ArgumentError) could not fetch environment variable "SENTRY_DSN" because it is not set
    (elixir 1.13.3) lib/system.ex:698: System.fetch_env!/1
    (stdlib 3.15) erl_eval.erl:685: :erl_eval.do_apply/6
    (stdlib 3.15) erl_eval.erl:893: :erl_eval.expr_list/6
    (stdlib 3.15) erl_eval.erl:237: :erl_eval.expr/5
    (stdlib 3.15) erl_eval.erl:229: :erl_eval.expr/5
    (stdlib 3.15) erl_eval.erl:893: :erl_eval.expr_list/6
    (stdlib 3.15) erl_eval.erl:408: :erl_eval.expr/5
-----> Fetching app dependencies with mix
** (ArgumentError) could not fetch environment variable "SENTRY_DSN" because it is not set
4

Can you confirm how you are accessing them?

I know secrets were not set because application failed because of missing env var and i checked the secrets page in dashboard.

Also, this problem may also be caused by changing region, before setting the secrets i also changed region of deployment

5

Also, this problem may also be caused by changing region, before setting the secrets i also changed region of deployment.

I did not change the region but I still get it. I’ll try to delete apps and then re-try. :smiley:

6

Env vars configured with flyctl secrets command are not available to builds. Check out how to set env vars for build

2 Likes
7

Ah, so there may be two different issues here. As @klucass says, those Fly secrets are available at run-time but the @andreyuhai output looks like build-time. Hence getting that error.

As for @morse it’s not clear if you are also getting an error at build-time or whether your error is at run-time. Secrets are globally available (you don’t need to set them per region) so it sounds like a build error for you too. But yes, certainly try again and see what happens.

8

I’ve been fighting with this bug as well. The “Activity” section of dashboard for a newly deployed app shows that the secrets were set, but the “Secrets” section is empty.

9

@ajbouh Maybe try running fly secrets list from the directory your app/fly.toml is in. Are the secrets listed there, using the CLI? If so, they are set. It’s just the UI is not showing them for some reason :thinking:.

Another way to check secrets is to SSH in to your app’s vm (with fly ssh console) and then check the value of one e.g echo $NAME (replacing that with the name of your secret, naturally). If your vm can show the value of the secret, the only way it could do that if they were set successfully. If not, well yep, that would confirm the secret was not being set.

10

The app crashes on deploy saying that the secrets aren’t set. That’s the observation I’m trusting the most. The empty “Secrets” panel is just supporting evidence.

11

I’m having the same issue with a GitHub Action to handle review environments (creating/deploying/destroying fly applications for PR branches).

  1. we create/launch a new application with flyctl launch [...] --no-deploy
  2. we set secrets (we’ve checked and secrets are correctly set) although we’re getting the same error Secrets are staged for the first deployment as the OP
  3. we do a first deploy, that is successful but secrets are empty.
  4. we set secrets again, exactly like step 2, and we now have secrets.

This really seems to be an issue with Fly, if anyone has any fix for this we would be glad.

Also, it’s very strange but when we deploy the app for the very first time (3) the version assigned by fly is v2 and we deploy it again (4) the version is v1.

1 Like
12

I’ve also seen issues where environment variables are not properly updated in a deployed image. I don’t know the pattern, but sometimes I see the environment variables I expect, and sometimes the vm that comes up is using old values. Very confusing.

13

I just deployed 2 apps setting secrets before the first deployment and both didn’t have any secret on the first run.

This is really bad UX, it means that you need to deploy 2 times to have the actual secrets and it makes things very confusing for new fly.io users that don’t know about this behaviour

14

I wonder if machine apps created with fly apps create work differently, because the secrets were available starting first deploy.

15

The commands i used are

fly apps create
fly ips allocate-v4 # because ips are not allocated for all my newly created apps, another fly issue
fly autoscale set min=1 max=1
fly regions set iad
fly secrets set --detach x=xxx
fly deploy --local-only ...
2 Likes