Rails 7+ applications do not need a `dockerfile-rails` gem

juanca · March 13, 2024, 7:44pm

Hello, I am using the latest Rails with Ruby 3.2.2. I noticed that flyctl is adding the dockerfile-rails gem to my Gemfile while launching. The latest Rails is already shipped with a Dockerfile.

Can the flyctl launch command stop adding a Dockerfile if it is already present and / or its the latest Rails?

My setup (while not perfect) does not allow for modifications to the Gemfile/Gemfile.lock during deployment.

rails@bfd0637e4614:/rails$ fly launch
Scanning source code
Detected a Rails app
Creating app in /rails
We're about to launch your Rails app on Fly.io. Here's what you're getting:

Organization: ***                                    (fly launch defaults to the personal org)
Name:         ***                                                 (derived from your directory name)
Region:       ***                             (this is the fastest region for you)
App Machines: shared-cpu-1x, 1GB RAM                                (most apps need about 1GB of RAM)
Postgres:     1 Node, shared-cpu-1x, 256MB RAM (1GB RAM), 10GB disk (determined from app source)
Redis:        <none>                                                (not requested)
Sentry:       false                                                 (not requested)

? Do you want to tweak these settings before proceeding? Yes
failed opening browser. Copy the url (https://fly.io/cli/launch/***) into a browser and continue
Waiting for launch data... Done
Created app '***' in organization 'personal'
Admin URL: https://fly.io/apps/***
Hostname: ***.fly.dev
Set secrets on ***: RAILS_MASTER_KEY
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies...
Retrying download gem from https://rubygems.org/ due to error (2/4): Bundler::PermissionError There was an error while trying to write to `/usr/local/bundle/cache/dockerfile-rails-1.6.6.gem`. It is likely that you need to grant write permissions for that path.
Retrying download gem from https://rubygems.org/ due to error (3/4): Bundler::PermissionError There was an error while trying to write to `/usr/local/bundle/cache/dockerfile-rails-1.6.6.gem`. It is likely that you need to grant write permissions for that path.
Retrying download gem from https://rubygems.org/ due to error (4/4): Bundler::PermissionError There was an error while trying to write to `/usr/local/bundle/cache/dockerfile-rails-1.6.6.gem`. It is likely that you need to grant write permissions for that path.
Bundler::PermissionError: There was an error while trying to write to
`/usr/local/bundle/cache/dockerfile-rails-1.6.6.gem`. It is likely that you need
to grant write permissions for that path.
  /usr/local/lib/ruby/3.2.0/bundler/shared_helpers.rb:105:in `rescue in
filesystem_access'
  /usr/local/lib/ruby/3.2.0/bundler/shared_helpers.rb:102:in `filesystem_access'
/usr/local/lib/ruby/3.2.0/bundler/rubygems_integration.rb:483:in `block in
download_gem'
  /usr/local/lib/ruby/3.2.0/bundler/retry.rb:40:in `run'
  /usr/local/lib/ruby/3.2.0/bundler/retry.rb:30:in `attempt'
/usr/local/lib/ruby/3.2.0/bundler/rubygems_integration.rb:474:in
`download_gem'
  /usr/local/lib/ruby/3.2.0/bundler/source/rubygems.rb:484:in `download_gem'
  /usr/local/lib/ruby/3.2.0/bundler/source/rubygems.rb:446:in `fetch_gem'
/usr/local/lib/ruby/3.2.0/bundler/source/rubygems.rb:430:in
`fetch_gem_if_possible'
  /usr/local/lib/ruby/3.2.0/bundler/source/rubygems.rb:158:in `install'
  /usr/local/lib/ruby/3.2.0/bundler/installer/gem_installer.rb:54:in `install'
/usr/local/lib/ruby/3.2.0/bundler/installer/gem_installer.rb:16:in
`install_from_spec'
/usr/local/lib/ruby/3.2.0/bundler/installer/parallel_installer.rb:156:in
`do_install'
/usr/local/lib/ruby/3.2.0/bundler/installer/parallel_installer.rb:147:in
`block in worker_pool'
  /usr/local/lib/ruby/3.2.0/bundler/worker.rb:62:in `apply_func'
  /usr/local/lib/ruby/3.2.0/bundler/worker.rb:57:in `block in process_queue'
  /usr/local/lib/ruby/3.2.0/bundler/worker.rb:54:in `loop'
  /usr/local/lib/ruby/3.2.0/bundler/worker.rb:54:in `process_queue'
/usr/local/lib/ruby/3.2.0/bundler/worker.rb:90:in `block (2 levels) in
create_threads'

An error occurred while installing dockerfile-rails (1.6.6), and
Bundler cannot continue.

In Gemfile:
  dockerfile-rails
Error: Failed to install dockerfile-rails gem, exiting: exit status 5

In the meantime, is it possible to launch and deploy an application without flyctl modifying the Gemfile?

juanca · March 13, 2024, 7:45pm

Added rails

juanca · March 13, 2024, 7:45pm

From General to Questions / Help

rubys · March 13, 2024, 8:03pm

Is there some way I can test for that?

Normally, bundle add will modify Gemfile.lock, so I’m puzzled why you are seeing this.

But more background. If you look at Rails 7.1’s Dockerfile, you will see my fingerprints all over it: Blaming rails/railties/lib/rails/generators/rails/app/templates/Dockerfile.tt at 7-1-stable · rails/rails · GitHub

What I have found is twofold: Rails’ approach of here’s an initial Dockerfile and after that you are on your own doesn’t serve the majority of the Rails developer community well. That and there are number of adaptations that are worth pursuing. As a concrete example: if you are using Postgres, you really would be better served running migrations before deploying to minimize downtime; this is true even if you have only one machine but is particularly true if you are running multiple machines.

The reason why the gem is installed is that in many cases the fix to problems people see is as simple as:

bin/rails generate dockerfile

And in most cases where that isn’t sufficient, adding a flag or two to the command is.

While I will work to find a way to accommodate your request, I don’t want to do so in a way that makes it more difficult for the many that this gem has helped.

P.S. You only need to run fly launch once. In fact, if you have a Dockerfile, can construct a fly.toml, all you need is fly apps create and can skip all of this. What I am leading to is that there is no reason why you can’t remove the gem now, and deploy.

juanca · March 13, 2024, 9:39pm

Yeah, my app is relatively new so it should be easy to set this up. I am on (latest) OS X using (latest) Docker Desktop.

Make a new Rails application with latest compatible ruby (3.2.2).
Copy the provided Dockerfile for a development environment. i.e. Dockerfile.dev
Modify Dockerfile.dev to avoid any production flags, precompilation e.g. it should run a development server (I can provide my copy if we need it)
Install flyctl after setting up the user in the Dockerfile
In the docker container: Launch a new fly.io application
See error

I think I did find that fly.io was providing the new Dockerfile. Thanks for the contributions!

rubys:

The reason why the gem is installed is that in many cases the fix to problems people see is as simple as:
bin/rails generate dockerfile
And in most cases where that isn’t sufficient, adding a flag or two to the command is.

While I will work to find a way to accommodate your request, I don’t want to do so in a way that makes it more difficult for the many that this gem has helped.

Is the reason why it helps people is because it generates a new Dockerfile? Or does it modify the existing Dockerfile?

I would bet that if the default Dockerfile provided by Rails works with fly.io then launching an application shouldn’t try to generate/overwrite the existing Dockerfile. If any complications arise then it is as simple as you say except with a new explicit step: 1) add gem; 2) bundle install; 3) bin/rails generate dockerfile (to explicitly overwrite application source code)

Oh, that’s nice. I do have the default Dockerfile provided by Rails. I just don’t know how to construct my own fly.toml I’ll look into this. Funny enough, when fly launch crashes in my OP, it seems to not create the fly.toml even though it configures the app in the fly.io dashboard.

juanca · March 13, 2024, 9:52pm

Here is my Dockerfile.dev (99% a copy from Dockerfile)

# syntax = docker/dockerfile:1

# Make sure RUBY_VERSION matches the Ruby version in .ruby-version and Gemfile
ARG RUBY_VERSION=3.2.2
FROM registry.docker.com/library/ruby:$RUBY_VERSION-slim as base

# Rails app lives here
WORKDIR /rails

# Throw-away build stage to reduce size of final image
FROM base as build

# Install packages needed to build gems
RUN apt-get update -qq && \
    apt-get install --no-install-recommends -y build-essential git libvips pkg-config

# Install application gems
COPY Gemfile Gemfile.lock ./
RUN bundle install

# Copy application code
COPY . .

# Final stage for app image
FROM base

# Install packages needed for deployment
RUN apt-get update -qq && \
    apt-get install --no-install-recommends -y curl libsqlite3-0 libvips && \
    rm -rf /var/lib/apt/lists /var/cache/apt/archives

# Copy built artifacts: gems, application
COPY --from=build /usr/local/bundle /usr/local/bundle
COPY --from=build /rails /rails

# Run and own only the runtime files as a non-root user for security
RUN useradd rails --create-home --shell /bin/bash && \
    chown -R rails:rails db log storage tmp
USER rails:rails

RUN curl -L https://fly.io/install.sh | sh
ENV PATH="$PATH:~/.fly/bin"

# Entrypoint prepares the database.
ENTRYPOINT ["/rails/bin/docker-entrypoint"]

# Start the server by default, this can be overwritten at runtime
EXPOSE 3000
CMD ["./bin/rails", "server"]

juanca · March 13, 2024, 9:55pm

Sweet, this worked. If someone in the future needs a Rails fly.toml:

app = <APP_NAME>
primary_region = 'sjc'
console_command = '/rails/bin/rails console'

[build]

[http_service]
  internal_port = 3000
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0
  processes = ['app']

[checks]
  [checks.status]
    port = 3000
    type = 'http'
    interval = '10s'
    timeout = '2s'
    grace_period = '5s'
    method = 'GET'
    path = '/up'
    protocol = 'http'
    tls_skip_verify = false

    [checks.status.headers]
      X-Forwarded-Proto = 'https'

[[vm]]
  size = 'shared-cpu-1x'

[[statics]]
  guest_path = '/rails/public'
  url_prefix = '/'

rubys · March 13, 2024, 10:49pm

Can I get you to run:

gem environment | grep INSTALL

If I can tell in advance that the installation directory is not writable, I will gladly skip this step.

More generally, I’m interested in your use case. While you have what you need for the moment, I would love to make it available to others via something like:

bin/rails generate dockerfile --dev

I suspect that if you truly want to create a dev box, you likely will want other things like openssh installed. Please let me know, and I can talk you through that.

Answers to other questions:

The dockerfile generator will detect if the file(s) it intends to create already exist, and will prompt you how to proceed. You can see the diffs, accept or reject the changes.
Telling someone whose first experience with fly.io is a failure that there is a multi-step procedure that might address their needs all too often results in customers deciding it isn’t worth the effort. For most customers, installing a gem is no big deal, and making their first experience as smooth as possible is a goal of mine. Of course, if installing a gem is destined to fail, trying anyway isn’t a great user experience, and I’d like to fix that.

juanca · March 13, 2024, 11:56pm

juanca@steyrpro % docker compose run -it app /bin/bash
rails@abe823bdb49e:/rails$ gem environment | grep INSTALL
  - INSTALLATION DIRECTORY: /usr/local/bundle
  - USER INSTALLATION DIRECTORY: /home/rails/.local/share/gem/ruby/3.2.0

Oh interesting. I might be interested in improving the dev experience. At the moment, I’ve only used render and fly.io for deploying my sample applications. I haven’t had a need to install more utilities. We can move this to another thread but I would be interested in understanding the use-cases for openssh etc.

My first application was way more vanilla and I just used a simple docker service:

services:
  app:
    image: ruby:3.2.2
    command: ./bin/rails server -b 0.0.0.0
    ports:
      - "3000:3000"
    volumes:
      - .:/root/app
      - fly:/root/.fly
      - bundle:/usr/local/bundle
    working_dir: "/root/app"
  selenium:
    image: selenium/standalone-chrome:latest
    ports:
      - "4444:4444"
      - "7900:7900"
    shm_size: "2g"
volumes:
  bundle:
  fly:

My second application to slightly* customize the image:

version: "3"
services:
  app:
    build:
      context: .
      dockerfile: Dockerfile.dev
    command: ./bin/rails server -b 0.0.0.0
    ports:
      - "3000:3000"
    volumes:
      - .:/rails
  selenium:
    image: selenium/standalone-chrome:latest
    ports:
      - "4444:4444"
      - "7900:7900"
    shm_size: "2g"

I think I remember this being the case with my first application – which didn’t have the bundle permission issue. I ended up throwing away the Gemfile and Dockerfile changes in favor of the defaults.

That makes sense to me. Thanks for thinking through the issues.

rubys · March 14, 2024, 12:28am

We have several threads going on here, I’m fine with keeping them together or splitting them out - doesn’t matter to me.

Can you verify that /home/rails/.local/share/gem/ruby/3.2.0 is writable?
If you are creating a development environment, you might want an IDE?
- Old instructions here: VS Code - Remote Server over SSH · Fly Docs
- Up to date (but only covering openssh) here: Run an ssh server · Fly Docs
I have a demo running docker compose you might be interested in: Demo running docker compose up on fly machines

I’m interested in exploring any or all of these, and updating code, guides, or other docs needed to help others with similar needs.

Edit: also possibly related: Generate a .devcontainer folder and its contents when creating a new … · rails/rails@c90a870 · GitHub

juanca · March 14, 2024, 12:56am

Looks like it is missing.

rails@51098c8d1229:/rails$ gem environment | grep INSTALL
  - INSTALLATION DIRECTORY: /usr/local/bundle
  - USER INSTALLATION DIRECTORY: /home/rails/.local/share/gem/ruby/3.2.0
rails@51098c8d1229:/rails$ ls -la /home/rails/
total 32
drwxr-xr-x 1 rails rails 4096 Mar 14 00:27 .
drwxr-xr-x 1 root  root  4096 Mar 14 00:06 ..
-rw-r--r-- 1 rails rails  220 Apr 23  2023 .bash_logout
-rw-r--r-- 1 rails rails 3526 Apr 23  2023 .bashrc
drwxr-xr-x 3 rails rails 4096 Mar 14 00:27 .bundle
drwxr-xr-x 6 rails rails 4096 Mar 14 00:39 .fly
-rw-r--r-- 1 rails rails  807 Apr 23  2023 .profile
rails@51098c8d1229:/rails$ ls -la /usr/local/bundle/
total 44
drwxrwxrwt  1 root root 4096 Mar 14 00:34 .
drwxr-xr-x  1 root root 4096 Jan 12 18:24 ..
drwxr-xr-x  2 root root 4096 Mar 14 00:06 bin
drwxr-xr-x  2 root root 4096 Mar 14 00:06 build_info
drwxr-xr-x  2 root root 4096 Mar 14 00:06 cache
drwxr-xr-x  2 root root 4096 Mar 14 00:06 doc
drwxr-xr-x  3 root root 4096 Mar 14 00:06 extensions
drwxr-xr-x 81 root root 4096 Mar 14 00:06 gems
drwxr-xr-x  2 root root 4096 Mar 14 00:06 plugins
drwxr-xr-x  2 root root 4096 Mar 14 00:06 specifications

I see. My current choice of IDE is Intellij IDEA. It doesn’t have remote development via SSH. But it does support a remote SDK (via docker) which has been good enough for me.

Oh, this is cool. I was thinking of doing some docker-in-docker in case I needed to scale cheaply. I opted for the Dockerfile.dev for now since my app is a proof-of-concept. Will revisit if I get traction.

Oh, wow! I can’t wait for this to be released. I am constantly copying and pasting between projects for the specific docker changes. It looks like the generated “Dockerfile.dev” is different from mine. I can see how that would be the root of my issues – especially since I am forcing the build command as the only way to install gems.

rubys · March 14, 2024, 6:22pm

Pull request created: Handle unwritable gem installation directories gracefully. by rubys · Pull Request #3370 · superfly/flyctl · GitHub

system · March 21, 2024, 6:22pm

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Preparations for Rails 7.1 rails	19	4605	January 17, 2023
Cut over to Rails Dockerfile Generator on Sunday 29 Jan 2023 announcement , rails	3	881	January 29, 2023
Fly launch for Rails is too invasive	5	279	December 14, 2023
Rails launch improvements (dockerfile-rails@1.6.7 w/ flyctl@0.2.22) Fresh Produce	0	154	March 19, 2024
Dockerless Rails Development rails	2	781	November 11, 2022

Rails 7+ applications do not need a `dockerfile-rails` gem

Related Topics