Environment variables

I tried with:

fly deploy --build-arg ZOOM_API_KEY="***" --build-arg ZOOM_API_SECRET="***"

and a few other variations, and I keep getting the same “key missing” issue.

The documentation lacks examples, or I couldn’t find them, and I’m not sure if the arguments are correctly built.

I’ll give this a try now, I think I still have a ported over app on Fly that’s failing at this build step issue. Will report back. Thanks @rubys!

@rubys that seems a good tip and I was able to move on and see that the server variables are there when performing the rails migration, however, it failed when running the server command. Maybe because the way I built the command:

[env]
  SERVER_COMMAND = "bin/rails fly:init && bundle exec puma"

How can I concatenate commands?

Oh, forget, my mistake … let me try again.

Did you also include

+ ARG ZOOM_API_KEY
+ ARG ZOOM_API_SECRET

in your dockerfile?

Ref: docker build with --build-arg with multiple arguments - Stack Overflow

Btw, docker doesn’t recommend using --build--arg for secrets, since the build command is visible in docker history.

I followed on from @rubys suggestion by trying:

namespace :fly do
  task init: "assets:precompile" do
    sh "bundle exec puma"
  end
end

Seems good so far, my build progressed right through to the auto migration step. Will take a look soon and see if it fully deployed.

Cheers both!

1 Like

Moving the precompile out of the Dockerfile seems to have raised a new issue…

Yarn executable was not detected in the system.

See below:

2022-08-28T21:45:04.022 app[34260aec] lhr [info] Starting clean up.
2022-08-28T21:45:11.103 runner[34260aec] lhr [info] Starting instance
2022-08-28T21:45:13.379 runner[34260aec] lhr [info] Configuring virtual machine
2022-08-28T21:45:13.388 runner[34260aec] lhr [info] Pulling container image
2022-08-28T21:45:13.972 runner[34260aec] lhr [info] Unpacking image
2022-08-28T21:45:13.989 runner[34260aec] lhr [info] Preparing kernel init
2022-08-28T21:45:14.414 runner[34260aec] lhr [info] Configuring firecracker
2022-08-28T21:45:14.594 runner[34260aec] lhr [info] Starting virtual machine
2022-08-28T21:45:14.783 app[34260aec] lhr [info] Starting init (commit: f815f49)...
2022-08-28T21:45:14.817 app[34260aec] lhr [info] Preparing to run: `/bin/bash -c ${SERVER_COMMAND}` as root
2022-08-28T21:45:14.862 app[34260aec] lhr [info] 2022/08/28 21:45:14 listening on [fdaa:0:8b99:a7b:bada:3426:aec:2]:22 (DNS: [fdaa::3]:53)
2022-08-28T21:45:19.845 app[34260aec] lhr [info] I, [2022-08-28T21:45:19.139996 #515] INFO -- : [SKYLIGHT] [5.3.3] Unable to start, see the Skylight logs for more details
2022-08-28T21:45:19.845 app[34260aec] lhr [info] I, [2022-08-28T21:45:19.404401 #515] INFO -- honeybadger: ** [Honeybadger] Initializing Honeybadger Error Tracker for Ruby. Ship it! version=4.12.2 framework=rails level=1 pid=515
2022-08-28T21:45:20.392 app[34260aec] lhr [info] yarn install v1.22.19
2022-08-28T21:45:20.516 app[34260aec] lhr [info] [1/4] Resolving packages...
2022-08-28T21:45:20.604 app[34260aec] lhr [info] success Already up-to-date.
2022-08-28T21:45:20.607 app[34260aec] lhr [info] Done in 0.22s.
2022-08-28T21:45:20.791 app[34260aec] lhr [info] yarn run v1.22.19
2022-08-28T21:45:20.814 app[34260aec] lhr [info] $ tailwindcss -i ./app/assets/stylesheets/application.tailwind.css -o ./app/assets/builds/application.css
2022-08-28T21:45:21.989 app[34260aec] lhr [info] Browserslist: caniuse-lite is outdated. Please run:
2022-08-28T21:45:21.989 app[34260aec] lhr [info] npx browserslist@latest --update-db
2022-08-28T21:45:21.989 app[34260aec] lhr [info] Why you should do it regularly: https://github.com/browserslist/browserslist#browsers-data-updating
2022-08-28T21:45:22.047 app[34260aec] lhr [info] warn - You have enabled the JIT engine which is currently in preview.
2022-08-28T21:45:22.047 app[34260aec] lhr [info] warn - Preview features are not covered by semver, may introduce breaking changes, and can change at any time.
2022-08-28T21:45:22.631 app[34260aec] lhr [info] Done in 649ms.
2022-08-28T21:45:22.644 app[34260aec] lhr [info] Done in 1.86s.
2022-08-28T21:45:22.783 app[34260aec] lhr [info] yarn install v1.22.19
2022-08-28T21:45:22.872 app[34260aec] lhr [info] [1/4] Resolving packages...
2022-08-28T21:45:22.964 app[34260aec] lhr [info] success Already up-to-date.
2022-08-28T21:45:22.968 app[34260aec] lhr [info] Done in 0.19s.
2022-08-28T21:45:23.125 app[34260aec] lhr [info] yarn run v1.22.19
2022-08-28T21:45:23.152 app[34260aec] lhr [info] $ esbuild app/javascript/*.* --bundle --outdir=app/assets/builds
2022-08-28T21:45:23.265 app[34260aec] lhr [info] app/assets/builds/application.js 177.5kb
2022-08-28T21:45:23.271 app[34260aec] lhr [info] Done in 0.15s.
2022-08-28T21:45:23.691 app[34260aec] lhr [info] Yarn executable was not detected in the system.
2022-08-28T21:45:23.691 app[34260aec] lhr [info] Download Yarn at https://yarnpkg.com/en/docs/install
2022-08-28T21:45:24.084 app[34260aec] lhr [info] Yarn executable was not detected in the system.
2022-08-28T21:45:24.084 app[34260aec] lhr [info] Download Yarn at https://yarnpkg.com/en/docs/install
2022-08-28T21:45:24.090 app[34260aec] lhr [info] rails aborted!
2022-08-28T21:45:24.090 app[34260aec] lhr [info] Command failed with exit 1: /usr/lib/fullstaq-ruby/versions/3.1.2-jemalloc/bin/ruby "/app/bin/yarn" install --immutable

You clearly have a yarn executable as it is found earlier in the process. Something in your build process must be running with a different PATH.

Not working for me either, but I’m having issues with the database:

PG::ConnectionBad
could not connect to server: Connection timed out
	Is the server running on host "*****" and accepting
	TCP/IP connections on port 5432?

After 3 days of trying everything, I might need to start from scratch

I’ll take a look at work tomorrow. Thanks for all your help so far.

Oh dear. Let me know if there is anything I can do to help. Suggestion for the future: tag topics related to rails with the rails tag as I have a watch on such topics. I saw rikki’s post first, and then followed him here.

Meanwhile, I’ve added Access to Environment Variables at Build Time to the docs. In there I’ve provided more detail on the tradeoffs involved in the solution.

On my machine with a fairly generic rails application with a Dockerfile generated by fly launch:

% fly ssh console -C "which yarn"
/root/.volta/bin/yarn

If you get similar results, one thing you might try is to add one line to your Dockerfile:

+ RUN for i in /root/.volta/bin/*; do ln -s $i /usr/local/bin; done
  COPY --from=gems /app /app
  COPY --from=node_modules /app/node_modules /app/node_modules

Thanks @rubys, I can’t verify where yarn lives due to not being able to ssh into a dead vm. However, your Dockerfile update didn’t seem to do the trick in my case unfortunately.

Is there any possibility of having the fly secrets available at build-time by default? Or is it too late for that on fly? You’re going to have a huge influx from Heroku, and at the moment, fly isn’t ready yet for an easy port across. I’ve not got a single app running on the platform yet.

I’m currently also evaluating Railway, Render and some other options but they too have their foibles when coming from Heroku… Fly is the leader in my opinion so far though.

That beyond my expertise at the moment, but at a minimum I assume that would not be a quick fix.

Perhaps you can get a VM that you can ssh into by doing the following:

  • removing assets:precompile (currently in your fly.rake file)
  • fly deploy --build-arg RAILS_ENV=development

When I do this with a new Rails application, attempting to access the server results in a Blocked Host with a link to Configuring Rails Applications — Ruby on Rails Guides, which is fine for the moment. Key is that the VM starts and I can ssh into it, and from there I can cd app and run bin/rails assets:precompile.

Another thing to try if all else fails and you simply want to ssh into the VM to debug.

Remove references that are preventing boot (i.e. assets:precompile)

Create hello.rb in your main app folder:

require 'rack'
Rack::Handler.default.run ->(env) { 
  [200, {"Content-Type" => "text/html"}, ["Hello World!"]] }, Port: 8080

In fly.toml, set SERVER_COMMAND = "bundle exec ruby hello.rb"

If you now deploy, you should be able to ssh into your VM and have plenty of RAM for rails console.

No, the platform goes through some trouble to ensure that secrets are only available to running instances of the app; our API can’t read them at all, and so we can’t deliver them to your local build.

1 Like

Thanks @thomas. I’m wondering about the remote build. Locally, developement is fine with local env vars. It’s just upon deployment, the Rails app chokes as it doesn’t have access to the project secret env vars.

I’ll re-phrase it slightly: Is there any possibility of having the fly secrets available at remote build-time by default?

I don’t think we’re likely to do that. It would involve brain surgery with the token-based system we use to distribute secrets, it would further expose secrets (remote builders are VMs that can be used to build multiple different apps — albeit in the same org), and we don’t want remote- and local- builds to be so drastically different from each other.

1 Like

I’ve posted a proposal that aims to make it easier to adjust when various steps are run for people new to fly.io: RFC: have fly launch produce a lib/tasks/fly.rake

Feedback welcome!

I’ve been playing around with commands that could solve the build ENV var problem—I think this works: fly deploy -e "$(fly ssh console -C 'printenv')"

This grabs the ENV vars from the running production instance, secrets and non-secrets, then dumps them into the build environment.

Note: this isn’t super secure. It’s recommended that you move your secrets into fly secrets and everything else into the [env] directive in your fly.toml file. Only use the command above to help troubleshoot builds.

1 Like