Initial deploy: ERROR [build 9/9] RUN SECRET_KEY_BASE=DUMMY ./bin/rails assets:precompile

I’m trying to get an initial deploy out for my rails app. I am not doing anything fancy, and following the basic steps from fly deploy. When doing so, I get:

------                                                                                                                                                                                                        
 > [build 9/9] RUN SECRET_KEY_BASE=DUMMY ./bin/rails assets:precompile:
#18 1.106 Missing encryption key to decrypt file with. Ask your team for your master key and write it to /rails/config/master.key or put it in the ENV['RAILS_MASTER_KEY'].
------
Error: failed to fetch an image or build from source: error building: executor failed running [/bin/sh -c SECRET_KEY_BASE=DUMMY ./bin/rails assets:precompile]: exit code: 1

Is there an easy fix to this (seemingly) popular issue? I can’t find an answer that works despite a lot of googling.

GEM
  remote: https://rubygems.org/
  specs:
    actioncable (7.0.5)
      actionpack (= 7.0.5)
      activesupport (= 7.0.5)
      nio4r (~> 2.0)
      websocket-driver (>= 0.6.1)
    actionmailbox (7.0.5)
      actionpack (= 7.0.5)
      activejob (= 7.0.5)
      activerecord (= 7.0.5)
      activestorage (= 7.0.5)
      activesupport (= 7.0.5)
      mail (>= 2.7.1)
      net-imap
      net-pop
      net-smtp
    actionmailer (7.0.5)
      actionpack (= 7.0.5)
      actionview (= 7.0.5)
      activejob (= 7.0.5)
      activesupport (= 7.0.5)
      mail (~> 2.5, >= 2.5.4)
      net-imap
      net-pop
      net-smtp
      rails-dom-testing (~> 2.0)
    actionpack (7.0.5)
      actionview (= 7.0.5)
      activesupport (= 7.0.5)
      rack (~> 2.0, >= 2.2.4)
      rack-test (>= 0.6.3)
      rails-dom-testing (~> 2.0)
      rails-html-sanitizer (~> 1.0, >= 1.2.0)
    actiontext (7.0.5)
      actionpack (= 7.0.5)
      activerecord (= 7.0.5)
      activestorage (= 7.0.5)
      activesupport (= 7.0.5)
      globalid (>= 0.6.0)
      nokogiri (>= 1.8.5)
    actionview (7.0.5)
      activesupport (= 7.0.5)
      builder (~> 3.1)
      erubi (~> 1.4)
      rails-dom-testing (~> 2.0)
      rails-html-sanitizer (~> 1.1, >= 1.2.0)
    activejob (7.0.5)
      activesupport (= 7.0.5)
      globalid (>= 0.3.6)
    activemodel (7.0.5)
      activesupport (= 7.0.5)
    activerecord (7.0.5)
      activemodel (= 7.0.5)
      activesupport (= 7.0.5)
    activestorage (7.0.5)
      actionpack (= 7.0.5)
      activejob (= 7.0.5)
      activerecord (= 7.0.5)
      activesupport (= 7.0.5)
      marcel (~> 1.0)
      mini_mime (>= 1.1.0)
    activesupport (7.0.5)
      concurrent-ruby (~> 1.0, >= 1.0.2)
      i18n (>= 1.6, < 2)
      minitest (>= 5.1)
      tzinfo (~> 2.0)
    addressable (2.8.4)
      public_suffix (>= 2.0.2, < 6.0)
    bindex (0.8.1)
    bootsnap (1.16.0)
      msgpack (~> 1.2)
    builder (3.2.4)
    bullet (7.0.7)
      activesupport (>= 3.0.0)
      uniform_notifier (~> 1.11)
    capybara (3.39.1)
      addressable
      matrix
      mini_mime (>= 0.1.3)
      nokogiri (~> 1.8)
      rack (>= 1.6.0)
      rack-test (>= 0.6.3)
      regexp_parser (>= 1.5, < 3.0)
      xpath (~> 3.2)
    coderay (1.1.3)
    concurrent-ruby (1.2.2)
    crass (1.0.6)
    date (3.3.3)
    debug (1.8.0)
      irb (>= 1.5.0)
      reline (>= 0.3.1)
    dockerfile-rails (1.4.1)
      rails
    dotenv (2.8.1)
    dotenv-rails (2.8.1)
      dotenv (= 2.8.1)
      railties (>= 3.2)
    erubi (1.12.0)
    faraday (2.7.5)
      faraday-net_http (>= 2.0, < 3.1)
      ruby2_keywords (>= 0.0.4)
    faraday-net_http (3.0.2)
    globalid (1.1.0)
      activesupport (>= 5.0)
    hashie (5.0.0)
    i18n (1.14.0)
      concurrent-ruby (~> 1.0)
    importmap-rails (1.1.6)
      actionpack (>= 6.0.0)
      railties (>= 6.0.0)
    io-console (0.6.0)
    irb (1.6.4)
      reline (>= 0.3.0)
    jbuilder (2.11.5)
      actionview (>= 5.0.0)
      activesupport (>= 5.0.0)
    jwt (2.7.0)
    loofah (2.21.3)
      crass (~> 1.0.2)
      nokogiri (>= 1.12.0)
    mail (2.8.1)
      mini_mime (>= 0.1.1)
      net-imap
      net-pop
      net-smtp
    marcel (1.0.2)
    matrix (0.4.2)
    method_source (1.0.0)
    mini_mime (1.1.2)
    minitest (5.18.0)
    msgpack (1.7.1)
    multi_xml (0.6.0)
    net-imap (0.3.4)
      date
      net-protocol
    net-pop (0.1.2)
      net-protocol
    net-protocol (0.2.1)
      timeout
    net-smtp (0.3.3)
      net-protocol
    nio4r (2.5.9)
    nokogiri (1.15.2-x86_64-darwin)
      racc (~> 1.4)
    nokogiri (1.15.2-x86_64-linux)
      racc (~> 1.4)
    oauth2 (2.0.9)
      faraday (>= 0.17.3, < 3.0)
      jwt (>= 1.0, < 3.0)
      multi_xml (~> 0.5)
      rack (>= 1.2, < 4)
      snaky_hash (~> 2.0)
      version_gem (~> 1.1)
    omniauth (2.1.1)
      hashie (>= 3.4.6)
      rack (>= 2.2.3)
      rack-protection
    omniauth-auth0 (3.1.0)
      omniauth (~> 2)
      omniauth-oauth2 (~> 1)
    omniauth-oauth2 (1.8.0)
      oauth2 (>= 1.4, < 3)
      omniauth (~> 2.0)
    omniauth-rails_csrf_protection (1.0.1)
      actionpack (>= 4.2)
      omniauth (~> 2.0)
    pg (1.5.3)
    pry (0.14.2)
      coderay (~> 1.1)
      method_source (~> 1.0)
    pry-rails (0.3.9)
      pry (>= 0.10.4)
    public_suffix (5.0.1)
    puma (5.6.5)
      nio4r (~> 2.0)
    racc (1.6.2)
    rack (2.2.7)
    rack-protection (3.0.6)
      rack
    rack-proxy (0.7.6)
      rack
    rack-test (2.1.0)
      rack (>= 1.3)
    rails (7.0.5)
      actioncable (= 7.0.5)
      actionmailbox (= 7.0.5)
      actionmailer (= 7.0.5)
      actionpack (= 7.0.5)
      actiontext (= 7.0.5)
      actionview (= 7.0.5)
      activejob (= 7.0.5)
      activemodel (= 7.0.5)
      activerecord (= 7.0.5)
      activestorage (= 7.0.5)
      activesupport (= 7.0.5)
      bundler (>= 1.15.0)
      railties (= 7.0.5)
    rails-dom-testing (2.0.3)
      activesupport (>= 4.2.0)
      nokogiri (>= 1.6)
    rails-html-sanitizer (1.6.0)
      loofah (~> 2.21)
      nokogiri (~> 1.14)
    railties (7.0.5)
      actionpack (= 7.0.5)
      activesupport (= 7.0.5)
      method_source
      rake (>= 12.2)
      thor (~> 1.0)
      zeitwerk (~> 2.5)
    rake (13.0.6)
    redis (4.8.1)
    regexp_parser (2.8.0)
    reline (0.3.5)
      io-console (~> 0.5)
    rexml (3.2.5)
    ruby2_keywords (0.0.5)
    rubyzip (2.3.2)
    selenium-webdriver (4.9.1)
      rexml (~> 3.2, >= 3.2.5)
      rubyzip (>= 1.2.2, < 3.0)
      websocket (~> 1.0)
    semantic_range (3.0.0)
    snaky_hash (2.0.1)
      hashie
      version_gem (~> 1.1, >= 1.1.1)
    sprockets (4.2.0)
      concurrent-ruby (~> 1.0)
      rack (>= 2.2.4, < 4)
    sprockets-rails (3.4.2)
      actionpack (>= 5.2)
      activesupport (>= 5.2)
      sprockets (>= 3.0.0)
    sqlite3 (1.6.3-x86_64-darwin)
    sqlite3 (1.6.3-x86_64-linux)
    stimulus-rails (1.2.1)
      railties (>= 6.0.0)
    tailwindcss-rails (2.0.29-x86_64-darwin)
      railties (>= 6.0.0)
    tailwindcss-rails (2.0.29-x86_64-linux)
      railties (>= 6.0.0)
    thor (1.2.2)
    timeout (0.3.2)
    turbo-rails (1.4.0)
      actionpack (>= 6.0.0)
      activejob (>= 6.0.0)
      railties (>= 6.0.0)
    tzinfo (2.0.6)
      concurrent-ruby (~> 1.0)
    uniform_notifier (1.16.0)
    version_gem (1.1.2)
    web-console (4.2.0)
      actionview (>= 6.0.0)
      activemodel (>= 6.0.0)
      bindex (>= 0.4.0)
      railties (>= 6.0.0)
    webdrivers (5.2.0)
      nokogiri (~> 1.6)
      rubyzip (>= 1.3.0)
      selenium-webdriver (~> 4.0)
    webpacker (5.4.4)
      activesupport (>= 5.2)
      rack-proxy (>= 0.6.1)
      railties (>= 5.2)
      semantic_range (>= 2.3.0)
    websocket (1.2.9)
    websocket-driver (0.7.5)
      websocket-extensions (>= 0.1.0)
    websocket-extensions (0.1.5)
    xpath (3.2.0)
      nokogiri (~> 1.8)
    zeitwerk (2.6.8)

PLATFORMS
  x86_64-darwin-21
  x86_64-linux

DEPENDENCIES
  bootsnap
  bullet
  capybara
  debug
  dockerfile-rails (>= 1.4)
  dotenv-rails
  importmap-rails
  jbuilder
  omniauth-auth0 (~> 3.0)
  omniauth-rails_csrf_protection (~> 1.0, >= 1.0.1)
  pg (~> 1.5)
  pry-rails
  puma (~> 5.0)
  rails (~> 7.0.3, >= 7.0.3.1)
  redis (~> 4.0)
  selenium-webdriver
  sprockets-rails
  sqlite3 (~> 1.4)
  stimulus-rails
  tailwindcss-rails (~> 2.0)
  turbo-rails
  tzinfo-data
  web-console
  webdrivers
  webpacker

RUBY VERSION
   ruby 3.1.0p0

BUNDLED WITH
   2.3.3

Normally with this error there is more log messages that give a clue. Perhaps try temporarily changing SECRET_KEY_BASE to RAILS_MASTER_KEY to see if you can coax out more error messages? You will need to change this back once you find and fix the real problem.

What’s going on here is that you need your master key to deploy, but not to build. Unfortunately the command to build your assets loads your configuration, and something inside your config files is attempting to decrypt something. Usually this means that there is a line that looks contains Rails.application.credentials in your config.

Here is what the output of fly deploy is:

Validating /REDACTED_PATH/fly.toml
Platform: machines
✓ Configuration is valid
--> Verified app config
==> Building image
Remote builder [REDACTED] ready
==> Creating build context
--> Creating build context done
==> Building image with Docker
--> docker host: 20.10.12 linux x86_64
[+] Building 0.9s (0/1)                                                                                                                                                                                       
[+] Building 10.8s (18/21)                                                                                                                                                                                    
 => [internal] load remote build context                                                                                                                                                                 0.0s
 => copy /context /                                                                                                                                                                                      0.7s
 => resolve image config for docker.io/docker/dockerfile:1                                                                                                                                               0.4s
 => CACHED docker-image://docker.io/docker/dockerfile:1@REDACTED                                                               0.0s
 => [internal] load metadata for docker.io/library/ruby:3.1.0-slim                                                                                                                                       0.1s
 => [base 1/3] FROM docker.io/library/ruby:3.1.0-slim@REDACTED 0.0s
 => CACHED [base 2/3] WORKDIR /rails                                                                                                                                                                     0.0s
 => CACHED [base 3/3] RUN gem update --system --no-document &&     gem install -N bundler                                                                                                                0.0s
 => CACHED [stage-2 1/4] RUN apt-get update -qq &&     apt-get install --no-install-recommends -y libsqlite3-0 postgresql-client &&     rm -rf /var/lib/apt/lists /var/cache/apt/archives                0.0s
 => CACHED [build 1/9] RUN apt-get update -qq &&     apt-get install --no-install-recommends -y build-essential curl libpq-dev node-gyp pkg-config python-is-python3                                     0.0s
 => CACHED [build 2/9] RUN curl -sL https://github.com/nodenv/node-build/archive/master.tar.gz | tar xz -C /tmp/ &&     /tmp/node-build-master/bin/node-build "19.6.1" /usr/local/node &&     npm insta  0.0s
 => CACHED [build 3/9] COPY --link Gemfile Gemfile.lock ./                                                                                                                                               0.0s
 => CACHED [build 4/9] RUN bundle install &&     bundle exec bootsnap precompile --gemfile &&     rm -rf ~/.bundle/ $BUNDLE_PATH/ruby/*/cache $BUNDLE_PATH/ruby/*/bundler/gems/*/.git                    0.0s
 => CACHED [build 5/9] COPY --link package.json yarn.lock ./                                                                                                                                             0.0s
 => CACHED [build 6/9] RUN yarn install --frozen-lockfile                                                                                                                                                0.0s
 => [build 7/9] COPY --link . .                                                                                                                                                                          0.2s
 => [build 8/9] RUN bundle exec bootsnap precompile app/ lib/                                                                                                                                            3.7s
 => ERROR [build 9/9] RUN SECRET_KEY_BASE=DUMMY ./bin/rails assets:precompile                                                                                                                            4.6s
------                                                                                                                                                                                                        
 > [build 9/9] RUN SECRET_KEY_BASE=DUMMY ./bin/rails assets:precompile:
#18 4.512 Missing encryption key to decrypt file with. Ask your team for your master key and write it to /rails/config/master.key or put it in the ENV['RAILS_MASTER_KEY'].
------
Error: failed to fetch an image or build from source: error building: executor failed running [/bin/sh -c SECRET_KEY_BASE=DUMMY ./bin/rails assets:precompile]: exit code: 1

Is there something more than this in terms of “more log messages that give a clue”? I am going to try to figure out how to do the other suggestion you made now :slight_smile:

That’s still SECRET_KEY_BASE. Can you change that to RAILS_MASTER_KEY in your Dockerfile and try redeploying?


The other way to approach this is to start with a blank slate:

rails new demo --main --minimal
cd demo
echo 'Rails.application.routes.draw {root "rails/welcome#index"}' \
  >> config/routes.rb
fly launch
fly deploy

With that working, copy first your Gemfile into this new directory, run bundle install, then fly deploy.

Next take a look at the files in your config directory and try copying them until you find which file causes the failure.

I was able to get the app to deploy by doing the following in my Dockerfile:

ENV RAILS_MASTER_KEY=[MASTER_KEY REDACTED]
RUN SECRET_KEY_BASE=[MASTER_KEY REDACTED] ./bin/rails assets:precompile

I will follow your steps to figure out which gem is the offender :slight_smile:

I’m a bit perplexed, and I may have done a poor job following your instructions, but I’ve done the following:

commented everything in my gem file except:

source "https://rubygems.org"

ruby "3.1.0"
gem "rails", "~> 7.0.3", ">= 7.0.3.1"
gem "bootsnap", require: false
gem "sprockets-rails"

only copied over the following config files:

application.rb
boot.rb
environment.rb
environments/development.rb
environments/production.rb
environments/test.rb.rb

As long as config.require_master_key = true exists in environments/production.rb, I get the error, even with this stripped down gemfile/config.

If I replace SECRET_KEY_BASE to RAILS_MASTER_KEY I get this error:

 > [build 9/9] RUN RAILS_MASTER_KEY=DUMMY ./bin/rails assets:precompile:
#18 1.050 rails aborted!
#18 1.050 ArgumentError: Missing `secret_key_base` for 'production' environment, set this string with `bin/rails credentials:edit`
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/application.rb:576:in `validate_secret_key_base'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/application.rb:419:in `secret_key_base'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/application.rb:157:in `key_generator'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/application.rb:181:in `message_verifier'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/activestorage-7.0.5/lib/active_storage/engine.rb:137:in `block (2 levels) in <class:Engine>'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/lazy_load_hooks.rb:92:in `block in execute_hook'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/lazy_load_hooks.rb:85:in `with_execution_control'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/lazy_load_hooks.rb:90:in `execute_hook'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/lazy_load_hooks.rb:76:in `block in run_load_hooks'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/lazy_load_hooks.rb:75:in `each'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/lazy_load_hooks.rb:75:in `run_load_hooks'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/application/finisher.rb:87:in `block in <module:Finisher>'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/initializable.rb:32:in `instance_exec'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/initializable.rb:32:in `run'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/initializable.rb:61:in `block in run_initializers'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/initializable.rb:60:in `run_initializers'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/application.rb:372:in `initialize!'
#18 1.050 /rails/config/environment.rb:5:in `<main>'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/bootsnap-1.16.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/zeitwerk-2.6.8/lib/zeitwerk/kernel.rb:38:in `require'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/application.rb:348:in `require_environment!'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/application.rb:506:in `block in run_tasks_blocks'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:188:in `invoke'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/sprockets-rails-3.4.2/lib/sprockets/rails/task.rb:61:in `block (2 levels) in define'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:243:in `block in invoke_prerequisites'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:241:in `each'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:241:in `invoke_prerequisites'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:218:in `block in invoke_with_call_chain'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:188:in `invoke'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:160:in `invoke_task'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block (2 levels) in top_level'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `each'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block in top_level'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:125:in `run_with_threads'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:110:in `top_level'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/commands/rake/rake_command.rb:24:in `block (2 levels) in perform'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:186:in `standard_exception_handling'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/commands/rake/rake_command.rb:24:in `block in perform'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/rake_module.rb:59:in `with_application'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/commands/rake/rake_command.rb:18:in `perform'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/command.rb:51:in `invoke'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/commands.rb:18:in `<main>'
#18 1.050 /rails/vendor/bundle/ruby/3.1.0/gems/bootsnap-1.16.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
#18 1.050 Tasks: TOP => environment
#18 1.050 (See full trace by running task with --trace)
------
Error: failed to fetch an image or build from source: error building: executor failed running [/bin/sh -c RAILS_MASTER_KEY=DUMMY ./bin/rails assets:precompile]: exit code: 1

Previously you were not getting a stack trace. I’d like to see a stack trace with SECRET_KEY_BASE=DUMMY, as the original Dockerfile was set up to do. If necessary set both .

ENV RAILS_MASTER_KEy=DUMMY
RUN SECRET_KEY_BASE=DUMMY ./bin/rails assets:precompile

If adding these files to the minimal application I suggested you start with exhibits this problem, and you were to post the contents of these files I will try to reproduce the problem and debug it. Feel free to redact the files, as long as the results still result in the failure you are seeing.

With:

I get:

 > [build 9/9] RUN SECRET_KEY_BASE=DUMMY ./bin/rails assets:precompile:                                                                                                                                       
#18 1.624 rails aborted!
#18 1.624 ActiveSupport::MessageEncryptor::InvalidMessage: ActiveSupport::MessageEncryptor::InvalidMessage
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/message_encryptor.rb:209:in `rescue in _decrypt'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/message_encryptor.rb:186:in `_decrypt'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/message_encryptor.rb:160:in `decrypt_and_verify'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/messages/rotator.rb:22:in `decrypt_and_verify'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/encrypted_file.rb:104:in `decrypt'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/encrypted_file.rb:66:in `read'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/encrypted_configuration.rb:43:in `read'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/encrypted_configuration.rb:64:in `config'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/encrypted_configuration.rb:79:in `options'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/core_ext/module/delegation.rb:303:in `method_missing'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activerecord-7.0.5/lib/active_record/railtie.rb:332:in `block in <class:Railtie>'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/initializable.rb:32:in `instance_exec'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/initializable.rb:32:in `run'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/initializable.rb:61:in `block in run_initializers'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/initializable.rb:60:in `run_initializers'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/application.rb:372:in `initialize!'
#18 1.624 /rails/config/environment.rb:5:in `<main>'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/bootsnap-1.16.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/zeitwerk-2.6.8/lib/zeitwerk/kernel.rb:38:in `require'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/application.rb:348:in `require_environment!'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/application.rb:506:in `block in run_tasks_blocks'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:188:in `invoke'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/sprockets-rails-3.4.2/lib/sprockets/rails/task.rb:61:in `block (2 levels) in define'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:243:in `block in invoke_prerequisites'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:241:in `each'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:241:in `invoke_prerequisites'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:218:in `block in invoke_with_call_chain'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:188:in `invoke'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:160:in `invoke_task'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block (2 levels) in top_level'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `each'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block in top_level'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:125:in `run_with_threads'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:110:in `top_level'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/commands/rake/rake_command.rb:24:in `block (2 levels) in perform'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:186:in `standard_exception_handling'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/commands/rake/rake_command.rb:24:in `block in perform'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/rake_module.rb:59:in `with_application'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/commands/rake/rake_command.rb:18:in `perform'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/command.rb:51:in `invoke'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/commands.rb:18:in `<main>'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/bootsnap-1.16.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
#18 1.624 
#18 1.624 Caused by:
#18 1.624 ArgumentError: key must be 16 bytes
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/message_encryptor.rb:196:in `key='
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/message_encryptor.rb:196:in `_decrypt'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/message_encryptor.rb:160:in `decrypt_and_verify'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/messages/rotator.rb:22:in `decrypt_and_verify'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/encrypted_file.rb:104:in `decrypt'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/encrypted_file.rb:66:in `read'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/encrypted_configuration.rb:43:in `read'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/encrypted_configuration.rb:64:in `config'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/encrypted_configuration.rb:79:in `options'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.5/lib/active_support/core_ext/module/delegation.rb:303:in `method_missing'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/activerecord-7.0.5/lib/active_record/railtie.rb:332:in `block in <class:Railtie>'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/initializable.rb:32:in `instance_exec'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/initializable.rb:32:in `run'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/initializable.rb:61:in `block in run_initializers'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/initializable.rb:60:in `run_initializers'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/application.rb:372:in `initialize!'
#18 1.624 /rails/config/environment.rb:5:in `<main>'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/bootsnap-1.16.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/zeitwerk-2.6.8/lib/zeitwerk/kernel.rb:38:in `require'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/application.rb:348:in `require_environment!'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/application.rb:506:in `block in run_tasks_blocks'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:188:in `invoke'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/sprockets-rails-3.4.2/lib/sprockets/rails/task.rb:61:in `block (2 levels) in define'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:243:in `block in invoke_prerequisites'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:241:in `each'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:241:in `invoke_prerequisites'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:218:in `block in invoke_with_call_chain'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/task.rb:188:in `invoke'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:160:in `invoke_task'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block (2 levels) in top_level'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `each'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block in top_level'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:125:in `run_with_threads'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:110:in `top_level'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/commands/rake/rake_command.rb:24:in `block (2 levels) in perform'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/application.rb:186:in `standard_exception_handling'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/commands/rake/rake_command.rb:24:in `block in perform'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/lib/rake/rake_module.rb:59:in `with_application'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/commands/rake/rake_command.rb:18:in `perform'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/command.rb:51:in `invoke'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/railties-7.0.5/lib/rails/commands.rb:18:in `<main>'
#18 1.624 /rails/vendor/bundle/ruby/3.1.0/gems/bootsnap-1.16.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
#18 1.624 Tasks: TOP => environment
#18 1.624 (See full trace by running task with --trace)
------
Error: failed to fetch an image or build from source: error building: executor failed running [/bin/sh -c SECRET_KEY_BASE=DUMMY ./bin/rails assets:precompile]: exit code: 1

Okay I was able to replicate, with a slight modification, I ran: rails new demo --minimal (left out --main as that was rails 7.1).

Here is the repo: GitHub - coreycondardo/fly-deployment-issue-demo

The only two additional files in config that are not in source control are credentials.yml.env and master.key.

When I deploy I get the same error I just posted in the previous message.

I was just having this error with a new rails app deployment.

I was able to workaround the problem by first commenting out the line below in the Dockerfile and then deploying the app. After the app deployed successfully, I removed the hashtag from the line and deployed again.

RUN SECRET_KEY_BASE=DUMMY ./bin/rails assets:precompile

The reason I had to deploy the app a second time was that even though it deployed the first time it wasn’t functional and was giving an error in the logs that it was missing some assets. Thankfully it is working correctly now.

FWIW for anyone coming here later wondering what was going on, rails has some weird requirements for requiring the master key during precompiling and even if you have it defined as a fly secret, it doesn’t have access to it during the specific build process. As hacky as my implementation may be, modifying my production.rb env file like this, works:

  if defined?(Rake) && Rake.respond_to?(:application) && Rake.application.respond_to?(:top_level_tasks)
    config.require_master_key = true if Rake.application.top_level_tasks.none? { |task| task.start_with?('assets:') }
  else
    config.require_master_key = true
  end

This basically tells rails to require the master key in all situations other than when any rake tasks involving assets are at play.

1 Like