Rails deployment is not ignoring :development gems (i.e. sqlite3)

Questions / Help
, , ,
1

I have SQLite and Postrgres broken out into :devlopment and :production sections (respectively) in my Gemfile. When I deploy, however, I get the following error. How do I get Fly to only install production gems. (When I deployed the same configuration to Heroku it worked).

 !
 !     Failed to install gems via Bundler.
 !     Detected sqlite3 gem which is not supported on Heroku:
 !     https://devcenter.heroku.com/articles/sqlite3

Gemfile:

group :production do
  gem "pg", "~> 1.1"
end

group :development do
   gem "sqlite3", "~>1.4"
end
2

I wasn’t able to reproduce this. Can you paste the full output of the build?

3

Hi @jsierles - here is the output. Also, at the bottom, I have my Gemfile posted.

Update available 0.0.303 -> v0.0.304.
Run "flyctl.exe version update" to upgrade.
==> Verifying app config
--> Verified app config
==> Building image
←[33mWARN ←[0mError connecting to local docker daemon: error during connect: This error may indicate that the docker daemon is not running.: Get "http://%2F%2F.%2Fpipe%2Fdocker_engine/_ping": open //./pipe/docker_engine: The system cannot find the file specified.
Remote builder fly-builder-empty-dew-5839 ready
==> Building image with Buildpacks
--> docker host: 20.10.12 linux x86_64
20: Pulling from heroku/buildpacks
Digest: sha256:65ae8bd7b834b60ec2539f61035b6343920b64a6e4984e367c1cd238782efdbc
Status: Image is up to date for heroku/buildpacks:20
20: Pulling from heroku/pack
Digest: sha256:d7fc54930aee3eb39efce2e0c03596fe8821b2be48e462b428264e6752134d01
Status: Image is up to date for heroku/pack:20
===> DETECTING
heroku/ruby     0.1.3
heroku/procfile 0.6.2
===> ANALYZING
Restoring metadata for "heroku/ruby:env" from app image
Restoring metadata for "heroku/ruby:gems" from app image
Restoring metadata for "heroku/ruby:ruby" from app image
===> RESTORING
Restoring data for "heroku/ruby:gems" from cache
===> BUILDING
-----> Installing bundler 2.2.21
-----> Removing BUNDLED WITH version in the Gemfile.lock
-----> Compiling Ruby/Rails
-----> Using Ruby version: ruby-3.0.3
-----> Loading Bundler Cache

###### WARNING:

       You have the `.bundle/config` file checked into your repository
        It contains local state like the location of the installed bundle
        as well as configured git local gems, and other settings that should
       not be shared between multiple checkouts of a single repo. Please
       remove the `.bundle/` folder from your repo and add it to your `.gitignore` file.

       https://devcenter.heroku.com/articles/bundler-configuration


###### WARNING:

       Removing `Gemfile.lock` because it was generated on Windows.
       Bundler will do a full resolve so native gems are handled properly.
       This may result in unexpected gem versions being used in your app.
       In rare occasions Bundler may not be able to resolve your dependencies at all.

       https://devcenter.heroku.com/articles/bundler-windows-gemfile

-----> Installing dependencies using bundler 2.2.21
       Running: BUNDLE_WITHOUT='development:test' BUNDLE_PATH=/layers/heroku_ruby/gems/vendor/bundle BUNDLE_BIN=vendor/bundle/bin bundle install -j4
       Fetching gem metadata from https://rubygems.org/...........
       Resolving dependencies......
       Using rake 13.0.6
       Using concurrent-ruby 1.1.9
       Using minitest 5.15.0
       Using builder 3.2.4
       Using erubi 1.10.0
       Using racc 1.6.0
       Using crass 1.0.6
       Using rack 2.2.3
       Using nio4r 2.5.8
       Using websocket-extensions 0.1.5
       Using marcel 1.0.2
       Using mini_mime 1.1.2
       Using digest 3.1.0
       Using io-wait 0.2.1
       Using timeout 0.2.0
       Using strscan 3.0.1
       Using public_suffix 4.0.6
       Using msgpack 1.4.5
       Using bundler 2.2.32
       Using matrix 0.4.2
       Using regexp_parser 2.2.1
       Using childprocess 4.1.0
       Using chronic 0.10.2
       Using io-console 0.5.11
       Using method_source 1.0.0
       Using thor 1.2.1
       Using ffi 1.15.5
       Using zeitwerk 2.5.4
       Using local_time 2.1.0
       Using pg 1.3.4
       Using rexml 3.2.5
       Using rubyzip 2.3.2
       Using tilt 2.0.10
       Fetching sqlite3 1.4.2
       Using i18n 1.10.0
       Using tzinfo 2.0.4
       Using nokogiri 1.13.3 (x86_64-linux)
       Using rack-test 1.1.0
       Using websocket-driver 0.7.5
       Using mail 2.7.1
       Using net-protocol 0.1.2
       Using addressable 2.8.0
       Using bootsnap 1.11.1
       Using puma 5.6.2
       Using sprockets 4.0.3
       Using reline 0.3.1
       Using sassc 2.4.0
       Using selenium-webdriver 4.1.0
       Using activesupport 7.0.2.3
       Using loofah 2.14.0
       Using net-imap 0.2.3
       Using net-pop 0.1.1
       Using net-smtp 0.3.1
       Using xpath 3.2.0
       Using whenever 1.0.0
       Using rails-dom-testing 2.0.3
       Using rails-html-sanitizer 1.4.2
       Using globalid 1.0.0
       Using activemodel 7.0.2.3
       Using capybara 3.36.0
       Using irb 1.4.1
       Using webdrivers 5.0.0
       Using actionview 7.0.2.3
       Using activerecord 7.0.2.3
       Using activejob 7.0.2.3
       Using debug 1.4.0
       Using actionpack 7.0.2.3
       Using jbuilder 2.11.5
       Using actioncable 7.0.2.3
       Using railties 7.0.2.3
       Using actionmailer 7.0.2.3
       Using activestorage 7.0.2.3
       Using sprockets-rails 3.4.2
       Using actionmailbox 7.0.2.3
       Using actiontext 7.0.2.3
       Using importmap-rails 1.0.3
       Using stimulus-rails 1.0.4
       Using turbo-rails 1.0.1
       Using sassc-rails 2.1.2
       Using rails 7.0.2.3
       Installing sqlite3 1.4.2 with native extensions
       Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

       current directory:
       /layers/heroku_ruby/gems/vendor/bundle/ruby/3.0.0/gems/sqlite3-1.4.2/ext/sqlite3
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/bin/ruby -I
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0 -r
       ./siteconf20220314-156-yycby2.rb extconf.rb
       checking for sqlite3.h... no
       sqlite3.h is missing. Try 'brew install sqlite3',
       'yum install sqlite-devel' or 'apt-get install libsqlite3-dev'
       and check your shared library search path (the
       location where your sqlite3 shared library is located).
       *** extconf.rb failed ***
       Could not create Makefile due to some reason, probably lack of necessary
       libraries and/or headers.  Check the mkmf.log file for more details.  You may
       need configuration options.

       Provided configuration options:
        --with-opt-dir
        --without-opt-dir
        --with-opt-include
        --without-opt-include=${opt-dir}/include
        --with-opt-lib
        --without-opt-lib=${opt-dir}/lib
        --with-make-prog
        --without-make-prog
        --srcdir=.
        --curdir
        --ruby=/layers/heroku_ruby/ruby/vendor/ruby-3.0.3/bin/$(RUBY_BASE_NAME)
        --with-sqlcipher
        --without-sqlcipher
        --with-sqlite3-config
        --without-sqlite3-config
        --with-pkg-config
        --without-pkg-config
        --with-sqlcipher
        --without-sqlcipher
        --with-sqlite3-dir
        --without-sqlite3-dir
        --with-sqlite3-include
        --without-sqlite3-include=${sqlite3-dir}/include
        --with-sqlite3-lib
        --without-sqlite3-lib=${sqlite3-dir}/lib

       To see why this extension failed to compile, please check the mkmf.log which can
       be found here:

       /layers/heroku_ruby/gems/vendor/bundle/ruby/3.0.0/extensions/x86_64-linux/3.0.0/sqlite3-1.4.2/mkmf.log

       extconf failed, exit code 1

       Gem files will remain installed in
       /layers/heroku_ruby/gems/vendor/bundle/ruby/3.0.0/gems/sqlite3-1.4.2 for
       inspection.
       Results logged to
       /layers/heroku_ruby/gems/vendor/bundle/ruby/3.0.0/extensions/x86_64-linux/3.0.0/sqlite3-1.4.2/gem_make.out

       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/rubygems/ext/builder.rb:93:in
       `run'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/rubygems/ext/ext_conf_builder.rb:47:in
       `block in build'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/tempfile.rb:317:in
       `open'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/rubygems/ext/ext_conf_builder.rb:26:in
       `build'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/rubygems/ext/builder.rb:159:in
       `build_extension'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/rubygems/ext/builder.rb:193:in
       `block in build_extensions'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/rubygems/ext/builder.rb:190:in
       `each'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/rubygems/ext/builder.rb:190:in
       `build_extensions'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/rubygems/installer.rb:845:in
       `build_extensions'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/rubygems_gem_installer.rb:71:in
       `build_extensions'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/rubygems_gem_installer.rb:28:in
       `install'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/source/rubygems.rb:200:in
       `install'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/installer/gem_installer.rb:54:in
       `install'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/installer/gem_installer.rb:16:in
       `install_from_spec'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/installer/parallel_installer.rb:186:in
       `do_install'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/installer/parallel_installer.rb:177:in
       `block in worker_pool'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/worker.rb:62:in
       `apply_func'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/worker.rb:57:in
       `block in process_queue'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/worker.rb:54:in
       `loop'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/worker.rb:54:in
       `process_queue'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/worker.rb:91:in
       `block (2 levels) in create_threads'

       An error occurred while installing sqlite3 (1.4.2), and Bundler cannot continue.

       In Gemfile:
         sqlite3
       Bundler Output: Fetching gem metadata from https://rubygems.org/...........
       Resolving dependencies......
       Using rake 13.0.6
       Using concurrent-ruby 1.1.9
       Using minitest 5.15.0
       Using builder 3.2.4
       Using erubi 1.10.0
       Using racc 1.6.0
       Using crass 1.0.6
       Using rack 2.2.3
       Using nio4r 2.5.8
       Using websocket-extensions 0.1.5
       Using marcel 1.0.2
       Using mini_mime 1.1.2
       Using digest 3.1.0
       Using io-wait 0.2.1
       Using timeout 0.2.0
       Using strscan 3.0.1
       Using public_suffix 4.0.6
       Using msgpack 1.4.5
       Using bundler 2.2.32
       Using matrix 0.4.2
       Using regexp_parser 2.2.1
       Using childprocess 4.1.0
       Using chronic 0.10.2
       Using io-console 0.5.11
       Using method_source 1.0.0
       Using thor 1.2.1
       Using ffi 1.15.5
       Using zeitwerk 2.5.4
       Using local_time 2.1.0
       Using pg 1.3.4
       Using rexml 3.2.5
       Using rubyzip 2.3.2
       Using tilt 2.0.10
       Fetching sqlite3 1.4.2
       Using i18n 1.10.0
       Using tzinfo 2.0.4
       Using nokogiri 1.13.3 (x86_64-linux)
       Using rack-test 1.1.0
       Using websocket-driver 0.7.5
       Using mail 2.7.1
       Using net-protocol 0.1.2
       Using addressable 2.8.0
       Using bootsnap 1.11.1
       Using puma 5.6.2
       Using sprockets 4.0.3
       Using reline 0.3.1
       Using sassc 2.4.0
       Using selenium-webdriver 4.1.0
       Using activesupport 7.0.2.3
       Using loofah 2.14.0
       Using net-imap 0.2.3
       Using net-pop 0.1.1
       Using net-smtp 0.3.1
       Using xpath 3.2.0
       Using whenever 1.0.0
       Using rails-dom-testing 2.0.3
       Using rails-html-sanitizer 1.4.2
       Using globalid 1.0.0
       Using activemodel 7.0.2.3
       Using capybara 3.36.0
       Using irb 1.4.1
       Using webdrivers 5.0.0
       Using actionview 7.0.2.3
       Using activerecord 7.0.2.3
       Using activejob 7.0.2.3
       Using debug 1.4.0
       Using actionpack 7.0.2.3
       Using jbuilder 2.11.5
       Using actioncable 7.0.2.3
       Using railties 7.0.2.3
       Using actionmailer 7.0.2.3
       Using activestorage 7.0.2.3
       Using sprockets-rails 3.4.2
       Using actionmailbox 7.0.2.3
       Using actiontext 7.0.2.3
       Using importmap-rails 1.0.3
       Using stimulus-rails 1.0.4
       Using turbo-rails 1.0.1
       Using sassc-rails 2.1.2
       Using rails 7.0.2.3
       Installing sqlite3 1.4.2 with native extensions
       Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

       current directory:
       /layers/heroku_ruby/gems/vendor/bundle/ruby/3.0.0/gems/sqlite3-1.4.2/ext/sqlite3
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/bin/ruby -I
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0 -r
       ./siteconf20220314-156-yycby2.rb extconf.rb
       checking for sqlite3.h... no
       sqlite3.h is missing. Try 'brew install sqlite3',
       'yum install sqlite-devel' or 'apt-get install libsqlite3-dev'
       and check your shared library search path (the
       location where your sqlite3 shared library is located).
       *** extconf.rb failed ***
       Could not create Makefile due to some reason, probably lack of necessary
       libraries and/or headers.  Check the mkmf.log file for more details.  You may
       need configuration options.

       Provided configuration options:
        --with-opt-dir
        --without-opt-dir
        --with-opt-include
        --without-opt-include=${opt-dir}/include
        --with-opt-lib
        --without-opt-lib=${opt-dir}/lib
        --with-make-prog
        --without-make-prog
        --srcdir=.
        --curdir
        --ruby=/layers/heroku_ruby/ruby/vendor/ruby-3.0.3/bin/$(RUBY_BASE_NAME)
        --with-sqlcipher
        --without-sqlcipher
        --with-sqlite3-config
        --without-sqlite3-config
        --with-pkg-config
        --without-pkg-config
        --with-sqlcipher
        --without-sqlcipher
        --with-sqlite3-dir
        --without-sqlite3-dir
        --with-sqlite3-include
        --without-sqlite3-include=${sqlite3-dir}/include
        --with-sqlite3-lib
        --without-sqlite3-lib=${sqlite3-dir}/lib

       To see why this extension failed to compile, please check the mkmf.log which can
       be found here:

       /layers/heroku_ruby/gems/vendor/bundle/ruby/3.0.0/extensions/x86_64-linux/3.0.0/sqlite3-1.4.2/mkmf.log

       extconf failed, exit code 1

       Gem files will remain installed in
       /layers/heroku_ruby/gems/vendor/bundle/ruby/3.0.0/gems/sqlite3-1.4.2 for
       inspection.
       Results logged to
       /layers/heroku_ruby/gems/vendor/bundle/ruby/3.0.0/extensions/x86_64-linux/3.0.0/sqlite3-1.4.2/gem_make.out

       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/rubygems/ext/builder.rb:93:in
       `run'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/rubygems/ext/ext_conf_builder.rb:47:in
       `block in build'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/tempfile.rb:317:in
       `open'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/rubygems/ext/ext_conf_builder.rb:26:in
       `build'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/rubygems/ext/builder.rb:159:in
       `build_extension'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/rubygems/ext/builder.rb:193:in
       `block in build_extensions'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/rubygems/ext/builder.rb:190:in
       `each'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/rubygems/ext/builder.rb:190:in
       `build_extensions'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/rubygems/installer.rb:845:in
       `build_extensions'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/rubygems_gem_installer.rb:71:in
       `build_extensions'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/rubygems_gem_installer.rb:28:in
       `install'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/source/rubygems.rb:200:in
       `install'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/installer/gem_installer.rb:54:in
       `install'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/installer/gem_installer.rb:16:in
       `install_from_spec'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/installer/parallel_installer.rb:186:in
       `do_install'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/installer/parallel_installer.rb:177:in
       `block in worker_pool'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/worker.rb:62:in
       `apply_func'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/worker.rb:57:in
       `block in process_queue'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/worker.rb:54:in
       `loop'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/worker.rb:54:in
       `process_queue'
       /layers/heroku_ruby/ruby/vendor/ruby-3.0.3/lib/ruby/3.0.0/bundler/worker.rb:91:in
       `block (2 levels) in create_threads'

       An error occurred while installing sqlite3 (1.4.2), and Bundler cannot continue.

       In Gemfile:
         sqlite3

 !
 !     Failed to install gems via Bundler.
 !     Detected sqlite3 gem which is not supported on Heroku:
 !     https://devcenter.heroku.com/articles/sqlite3
 !
ERROR: failed to build: exit status 1
Error failed to fetch an image or build from source: executing lifecycle: failed with status code: 51

Gemfile:

source "https://rubygems.org"
git_source(:github) { |repo| "https://github.com/#{repo}.git" }

ruby "3.0.3"

# Bundle edge Rails instead: gem "rails", github: "rails/rails", branch: "main"
gem "rails", "~> 7.0.1"

# The original asset pipeline for Rails [https://github.com/rails/sprockets-rails]
gem "sprockets-rails"

# Use the Puma web server [https://github.com/puma/puma]
gem "puma", "~> 5.0"

# Use JavaScript with ESM import maps [https://github.com/rails/importmap-rails]
gem "importmap-rails"

# Hotwire's SPA-like page accelerator [https://turbo.hotwired.dev]
gem "turbo-rails"

# Hotwire's modest JavaScript framework [https://stimulus.hotwired.dev]
gem "stimulus-rails"

# Build JSON APIs with ease [https://github.com/rails/jbuilder]
gem "jbuilder"

# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
gem "tzinfo-data", platforms: %i[ mingw mswin x64_mingw jruby ]

# Reduces boot times through caching; required in config/boot.rb
gem "bootsnap", require: false

# Use Sass to process CSS
gem "sassc-rails"

# Cron Jobs
gem 'whenever', '~> 1.0'

gem 'local_time', '~> 2.1.0'

group :production do
  gem "pg", "~> 1.1"
end

group :development, :test do
  # See https://guides.rubyonrails.org/debugging_rails_applications.html#debugging-with-the-debug-gem
  gem "debug", platforms: %i[ mri mingw x64_mingw ]

  # Use sqlite3 as the database for Active Record
  gem "sqlite3", "~> 1.4"
end

group :development do
  # Use console on exceptions pages [https://github.com/rails/web-console]
  gem "web-console"
end

group :test do
  # Use system testing [https://guides.rubyonrails.org/testing.html#system-testing]
  gem "capybara"
  gem "selenium-webdriver"
  gem "webdrivers"
end
4

There’s a message here saying: You have the .bundle/config file checked into your repository

What’s in that file? Can you try removing it before deploying? That file may need to be in .dockerignore if it’s also in .gitignore.

5

I think that may have been the ticket @jsierles. That file had BUNDLE_WITHOUT "production:development".

Interestingly, .bundle was already being ignored in the .dockerignore (contents of .dockerignore below):

.git
tmp
log
public/assets
.bundle
6

Ah, which version of flyctl? Until version 3.0.4, .dockerignore would not apply correctly to buildpack-based builds.

7

flyctl version produces 0.0.303. I am assuming that is version 3.0.3. Updating now.

8

Hello everyone I am having the same problem mentioned in this post, the only way I could get the fly deploy command to succeed was by extracting the sqlite3 gem from the development/test group and put it outside

gem "sqlite3", "1.4.2"

group :development, :test do
  #gem "sqlite3", "1.4.2"
  gem "debug",   "1.5.0", platforms: %i[ mri mingw x64_mingw ]
end

But this shouldn’t be the correct way to fix this, don´t you think?