With an official Dockerfile
landing in Rails main branch (rails/Dockerfile.tt at main · rubys/rails · GitHub), I’d like to propose a scheme in which Docker apt-get package dependencies are declared per gem in a .gemspec
to improve the Docker deployment story now and in the future.
Why
At the time of this writing, packages for Docker are resolved via this code in main (Rails 7.1, which isn’t released yet): rails/app_base.rb at 0e23b0427e86aac5171d02912e3692427c79b800 · rubys/rails · GitHub. This works great for most Rails installations, but when a gem is added that depends on a package being present in a ruby:#{RUBY_VERSION}-slim
that’s not present, it must be manually added. Under those circumstances, gem maintainers usually end up documenting packages in a README, like Nokogiri does at Installing Nokogiri - Nokogiri (note that Nokogiri installs in the default Rails Dockerfile).
Providing a list of apt-get
packages in gemspec.metadata = {"ruby_slim_docker_packages" => "foo bar"}
could eliminate the need for developers to dig through documentation to find the right packages to deploy their Rails application.
The proposal
RubyGems could declared their ruby:#{RUBY_VERSION}-slim
package dependencies via the ruby_slim_docker_packages
key in the gemspec.metadata
.
For example, the active storage
gem relies on the libvips
package to run in the ruby:3.2.0-slim
image. Here’s what the final gemspec would look like (see the ):
diff --git a/activestorage/activestorage.gemspec b/activestorage/activestorage.gemspec
index 79f6cc50f9..6d8553c750 100644
--- a/activestorage/activestorage.gemspec
+++ b/activestorage/activestorage.gemspec
@@ -27,6 +27,7 @@
"mailing_list_uri" => "https://discuss.rubyonrails.org/c/rubyonrails-talk",
"source_code_uri" => "https://github.com/rails/rails/tree/v#{version}/activestorage",
"rubygems_mfa_required" => "true",
+ "ruby_slim_docker_packages" => "libvips"
}
When a Dockerfile is generated, it could inflect on the Bundle and extract the packages as follows:
def dockerfile_packages
Bundler.load.specs
.map{ |gem| gem.metadata["ruby_slim_docker_packages"] }
.compact
.map{ |packages| packages.split(" ") }
.flatten
.unique
end
Additional, a rails dockerfile:packages
command could be implemented that lists the output of the function above.
The following gems would have the following packages declared in the ruby_slim_docker_packages
metadata:
-
rails
-build-essential git
-
mysql2
-default-libmysqlclient-dev
-
pg
-libpq-dev
-
mysql2
-default-libmysqlclient-dev
Arguments against
Package management has a lot of inconsistencies between platforms, versions, and operating systems. The dream would be to support more platforms outside of ruby:slim
, like brew packages; however that could get really complicated and require a more sophisticated resolution mechanism than what metadata could provide.
Here’s what the current situation looks like for installing node dependencies in the ruby:#{RUBY_VERSION}-slim
packages:
def dockerfile_packages
# start with the essentials
packages = %w(build-essential git)
# add databases: sqlite3, postgres, mysql
packages += %w(pkg-config libpq-dev default-libmysqlclient-dev)
# add redis in case Action Cable, caching, or sidekiq are added later
packages << "redis"
# ActiveStorage preview support
packages << "libvips" unless skip_active_storage?
# node support, including support for building native modules
if using_node?
packages += %w(curl node-gyp) # pkg-config already listed above
# module build process depends on Python, and debian changed
# how python is installed with the bullseye release. Below
# is based on debian release included with the Ruby images on
# Dockerhub.
case Gem.ruby_version
when /^2.7/
bullseye = ruby_version >= "2.7.4"
when /^3.0/
bullseye = ruby_version >= "3.0.2"
else
bullseye = true
end
if bullseye
packages << "python-is-python3"
else
packages << "python"
end
end
packages.sort
This logic could live in the jsbundling-rails
gemspec (or whichever gem handles JS bundling), but as you can see, this could get complicated as OS maintainers change packages.
Because of this complexity and the unknowns for how useful and widely adopted this scheme could be, I propose keeping the target platform narrow to the ruby:slim
image to see if it actually improves the deployment story.
Arguments for
As people add gems to their Rails applications, it would save them time from digging through deployment documentation if they can instead run a command that lists out the packages they need to run in the upcoming Rails Dockerfile.
Moving gem package dependencies from Rails into each gem would empower gem maintainers with a way to provide their users with a better Rails deployment experience. We’d hopefully see more PRs opened in the community to help gem maintainers automate and reason through package dependencies.
For more complex deployments, like including Python + Chrome to install a nodejs env in the image via multiple Docker build steps, we might see somebody from the Ruby community with knowledge on maintaining OS packages move this complexity into a package, which could further improve the build & deployment experience.
Discussion points
To be clear, I think the current approach of hardcoding packages into the current Rails Dockerfile generator is necessary because gems don’t yet have the metadata needed to declare package dependencies. An effort by the community would be needed to get this working properly and a lot of gems would have to be updated.
My knowledge of maintaining Linux packages is limited. I’d be curious to hear from people who have to maintain Linux systems what problems they’d anticipate from such a scheme. My hopes are that this proposal helps sysadmins work closer with Ruby developers via collaborations on what should go into the ruby_slim_docker_packages
metadata.
The longer-term dream would be getting something like this working for more platforms. For example, a gem could specify Brew dependencies for macOS development environments. This feels like it would be too complicated for a place to start, but you can see how this might improve setting up a development environment on macOS if a list of brew packages can be pulled from Gem manifests.
The biggest question of all: who would find this useful and why?