The command fly deploy fails while bundling I believe: => ERROR [gems 2/2] RUN bundle install && rm -rf vendor/bundle/ruby/*/cache
I don’t see the same error details on this forum. When I try to investigate with fly logs it just hangs and doesn’t do anything. I’m using Rails in API mode with React frontend… Ruby version 2.7.6p219 and Rails version 6.1.7
I so appreciate the help! I’ve burned so many hours without forward progress.
I’m having trouble reproducing this (as in, when I try nokogiri installs fine). That being said, based on the error message I’d suggest that you edit your Dockerfile, look for the line that starts with ARG BUILD_PACKAGES, and add pkg-config libxml2-dev libxslt-dev to the list. See Installing Nokogiri - Nokogiri for details.
If you haven’t already, edit lib/tasks/fly.rake', and remove => 'assets:precompile’fromtask :build` as you have no assets to precompile.
@rubys
Thank you so much for your reply.
Unfortunately I get the same error when I make the Dockerfile change. (I already had the fly.rake edit you suggested)
Do you know where I can find the mkmf.log file?
I am also going to paste my Dockerfile here if that helps understand the problem…
Thank you!
# syntax = docker/dockerfile:experimental
# Dockerfile used to build a deployable image for a Rails application.
# Adjust as required.
#
# Common adjustments you may need to make over time:
# * Modify version numbers for Ruby, Bundler, and other products.
# * Add library packages needed at build time for your gems, node modules.
# * Add deployment packages needed by your application
# * Add (often fake) secrets needed to compile your assets
#######################################################################
# Learn more about the chosen Ruby stack, Fullstaq Ruby, here:
# https://github.com/evilmartians/fullstaq-ruby-docker.
#
# We recommend using the highest patch level for better security and
# performance.
ARG RUBY_VERSION=2.7.6
ARG VARIANT=jemalloc-slim
FROM quay.io/evl.ms/fullstaq-ruby:${RUBY_VERSION}-${VARIANT} as base
LABEL fly_launch_runtime="rails"
ARG BUNDLER_VERSION=2.3.17
ARG RAILS_ENV=production
ENV RAILS_ENV=${RAILS_ENV}
ENV RAILS_SERVE_STATIC_FILES true
ENV RAILS_LOG_TO_STDOUT true
ARG BUNDLE_WITHOUT=development:test
ARG BUNDLE_PATH=vendor/bundle
ENV BUNDLE_PATH ${BUNDLE_PATH}
ENV BUNDLE_WITHOUT ${BUNDLE_WITHOUT}
RUN mkdir /app
WORKDIR /app
RUN mkdir -p tmp/pids
RUN gem update --system --no-document && \
gem install -N bundler -v ${BUNDLER_VERSION}
#######################################################################
# install packages only needed at build time
FROM base as build_deps
ARG BUILD_PACKAGES="git build-essential libpq-dev wget vim curl gzip xz-utils libsqlite3-dev pkg-config libxml2-dev libxslt-dev"
ENV BUILD_PACKAGES ${BUILD_PACKAGES}
RUN --mount=type=cache,id=dev-apt-cache,sharing=locked,target=/var/cache/apt \
--mount=type=cache,id=dev-apt-lib,sharing=locked,target=/var/lib/apt \
apt-get update -qq && \
apt-get install --no-install-recommends -y ${BUILD_PACKAGES} \
&& rm -rf /var/lib/apt/lists /var/cache/apt/archives
#######################################################################
# install gems
FROM build_deps as gems
COPY Gemfile* ./
RUN bundle install && rm -rf vendor/bundle/ruby/*/cache
#######################################################################
# install deployment packages
FROM base
ARG DEPLOY_PACKAGES="postgresql-client file vim curl gzip libsqlite3-0"
ENV DEPLOY_PACKAGES=${DEPLOY_PACKAGES}
RUN --mount=type=cache,id=prod-apt-cache,sharing=locked,target=/var/cache/apt \
--mount=type=cache,id=prod-apt-lib,sharing=locked,target=/var/lib/apt \
apt-get update -qq && \
apt-get install --no-install-recommends -y \
${DEPLOY_PACKAGES} \
&& rm -rf /var/lib/apt/lists /var/cache/apt/archives
# copy installed gems
COPY --from=gems /app /app
COPY --from=gems /usr/lib/fullstaq-ruby/versions /usr/lib/fullstaq-ruby/versions
COPY --from=gems /usr/local/bundle /usr/local/bundle
#######################################################################
# Deploy your application
COPY . .
# Adjust binstubs to run on Linux and set current working directory
RUN chmod +x /app/bin/* && \
sed -i 's/ruby.exe\r*/ruby/' /app/bin/* && \
sed -i 's/ruby\r*/ruby/' /app/bin/* && \
sed -i '/^#!/aDir.chdir File.expand_path("..", __dir__)' /app/bin/*
# The following enable assets to precompile on the build server. Adjust
# as necessary. If no combination works for you, see:
# https://fly.io/docs/rails/getting-started/existing/#access-to-environment-variables-at-build-time
ENV SECRET_KEY_BASE 1
# ENV AWS_ACCESS_KEY_ID=1
# ENV AWS_SECRET_ACCESS_KEY=1
# Run build task defined in lib/tasks/fly.rake
ARG BUILD_COMMAND="bin/rails fly:build"
RUN ${BUILD_COMMAND}
# Default server start instructions. Generally Overridden by fly.toml.
ENV PORT 8080
ARG SERVER_COMMAND="bin/rails fly:server"
ENV SERVER_COMMAND ${SERVER_COMMAND}
CMD ${SERVER_COMMAND}
Can you also post your Gemfile? With that and your Dockerfile I should be able to reproduce the problem here.
mkmf.log is on the vm which fails to deploy; making it inaccessible. For really hard problems, I add the following to lib/tasks/fly.rake:
require 'rack'
require 'rack/handler/puma'
namespace :mock do
desc 'Mock server - useful for debugging startup issues'
task :server do
handler = Rack::Handler::Puma
class RackApp
def call(env)
[200, {"Content-Type" => "text/plain"}, ["Hello from Fly.io"]]
end
end
handler.run RackApp.new, Port: ENV['PORT'] || 8080
end
end
Then change SERVER_COMMAND:
ARG SERVER_COMMAND="bin/rails mock:server"
And finally comment out any step that fails. This way the VM will start, I can run flyctl ssh console and run the commands that fail for my self and observe the results.
When I commented out the line RUN bundle install && rm -rf vendor/bundle/ruby/*/cache
from the Dockerfile, the build did progress further along, but then failed at => ERROR [stage-3 7/7] RUN bin/rails fly:build - I believe because none of the dependences were installed due to the above line being commented out. So I think this is progress…
I still don’t get what is going on, however. Appreciate your help.
My Gemfile is below.
source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }
ruby "2.7.6"
# Bundle edge Rails instead: gem 'rails', github: 'rails/rails', branch: 'main'
gem 'rails', '~> 6.1.3', '>= 6.1.3.2'
# Use postgresql as the database for Active Record
gem 'pg', '~> 1.1'
# Use Puma as the app server
gem 'puma', '~> 5.0'
# Build JSON APIs with ease. Read more: https://github.com/rails/jbuilder
# gem 'jbuilder', '~> 2.7'
# Use Redis adapter to run Action Cable in production
gem 'redis', '~> 4.0'
# Use Active Model has_secure_password
gem 'bcrypt', '~> 3.1.7'
gem 'faker'
gem 'nokogiri', '1.11.7'
# Use Active Storage variant
# gem 'image_processing', '~> 1.2'
# Reduces boot times through caching; required in config/boot.rb
gem 'bootsnap', '>= 1.4.4', require: false
# Use Rack CORS for handling Cross-Origin Resource Sharing (CORS), making cross-origin AJAX possible
# The below has been uncommented:
gem 'rack-cors'
group :development, :test do
# Call 'byebug' anywhere in the code to stop execution and get a debugger console
gem 'byebug', platforms: [:mri, :mingw, :x64_mingw]
end
group :development do
gem 'listen', '~> 3.3'
# Spring speeds up development by keeping your application running in the background. Read more: https://github.com/rails/spring
gem 'spring'
end
# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
gem 'tzinfo-data', platforms: [:mingw, :mswin, :x64_mingw, :jruby]
group :development, :test do
gem 'rspec-rails', '~> 5.0.0'
end
group :test do
gem 'rspec-json_expectations'
gem 'shoulda-matchers', '~> 4.0'
end
gem "active_model_serializers", "~> 0.10.12"
Sadly, this builds and deploys without error for me.
The next place to look is Gemfile.lock. If you are willing, delete that file. If the specific versions listed in that file will be important to you later, simply move that file outside of your application temporarily.
It also might be useful to see if starting with an empty app (like I did) works for you.
Create a new rails app (rails new testapp). Just so the app does something, add root "rails/welcome#index" to config/routes.rb. Copy your Dockerfile, Gemfile, fly.toml, and lib/tasks/fly.rake from your existing app to the new app. Delete your Gemfile.lock in the new app.
@rubys Your suggestion to delete the Gemfile.lock corrected the problem (at least for the Rails API). Thank you! I don’t know why, but somehow my Gemfile.lock was derailing the build process. I must have done something earlier to mess it up. Once deleted, fly deploy will run to completion.
Now I can make API requests, but I still am not getting my React frontend (create react app) to work. Are there some Rake tasks or Dockerfile settings I can make to run this?
Can Fly be used to run both in one project? Apologies if this is basic stuff. This is how I used to do it at Heroku. Or do I need to use a service like Vercel or Netlify to host the frontend? Of course I’d rather deploy them together to simplify and avoid CORS issues…
Again, I’m using Rails in API mode and a React Frontend within a /client subfolder.
I would be very curious if the problem comes back if you run bundle install locally and then deploy. If so, let’s debug further.
Before making any changes, I would encourage you to take advantage of the fact that you have a running vm to explore. You should be able to fly ssh console to access the machine. See if you can get your client to start. Hopefully this is just a matter of cd’ing into the right directory, ensuring your PATH is set correctly (check the Dockerfile to see how it should be set), and then running a command like 'npm run clientor perhapsyarn client`.
Check your client/package.json for a proxy setting. If it is present, you want rails starting on that port and your client starting on the port mentioned as internal_port in your fly.toml.
The final step is to get both the node client and rails api server started. Two approaches I’ve used, pick the one you prefer:
There is an npm module called concurrently that you can install and add to your package.json in a new script. It can start both processes.
There is a gem called foreman that can run a Procfile that you can create to run both processes.
Now change the SERVER_COMMAND to run either the node script or foreman based on your preference. Note that this is defined in your Dockerfile but overrridden in your fly.toml. Be sure the correct value is in the fly.toml.
To answer your first question, after I deleted the Gemfile.lock and initially deployed, then ran bundle install I could run subsequent fly deploys with no trouble. So there must have been a problem with the file. Likely I had previously updated something incorrectly.
Question on Ports:
There are ports listed in the Docker file: ENV PORT 8080
and the fly.toml file: [env] PORT = "8080" and internal_port = 8080
Are these all referring to the same thing? Or is one for what the public is routed to (should be the React frontend) and the other for where the Rails API will be located?
I started messing with these and got the “Failed due to unhealthy allocations” error, which I believe is due to the Ports being incorrect.
Correct. If fly deploy doesn’t detect an application listening on the internal_port, it will treat the deployment as having failed.
Rails defaults to port 3000. The ENV in the Dockerfile will override that. And the [env] in the fly.toml will override the Dockerfile.
No. React can be set up to proxy requests to the Rails API server, and if you are configured this way, from Fly.io’s perspective you will only be defining a single port, and that port will be the one that is listened to by the React frontend.
Another way to configure this is without a proxy, and in such a case you would have two separate [[services]] sections, one each for the React frontend and the Rails API server.
I again encourage you to verify that you can start the React frontend before attempting to adjust the port.
Hi @rubys I was able to start the React frontend, but only after fly ssh console 'ing in and manually installing node and npm. Then I could start React.
How would I do this automatically with the Dockerfile or the fly.toml file? Or is there a good resource that explains this? Thank you for your ongoing help!
I’m actually working on the documentation you are asking for as we speak. Meanwhile, following is what fly launch would have generated for you, had it detected that you required node.js. Replace the lines in your Dockerfile that start with LABEL and and end with ‘# install deployment packages’. Adjust the ARG *_VERSION lines to match your environment:
LABEL fly_launch_runtime="rails"
ARG NODE_VERSION=19.3.0
ARG YARN_VERSION=1.22.19
ARG BUNDLER_VERSION=2.3.26
ARG RAILS_ENV=production
ENV RAILS_ENV=${RAILS_ENV}
ENV RAILS_SERVE_STATIC_FILES true
ENV RAILS_LOG_TO_STDOUT true
ARG BUNDLE_WITHOUT=development:test
ARG BUNDLE_PATH=vendor/bundle
ENV BUNDLE_PATH ${BUNDLE_PATH}
ENV BUNDLE_WITHOUT ${BUNDLE_WITHOUT}
RUN mkdir /app
WORKDIR /app
RUN mkdir -p tmp/pids
ENV VOLTA_HOME /root/.volta
ENV PATH $VOLTA_HOME/bin:/usr/local/bin:$PATH
RUN gem update --system --no-document && \
gem install -N bundler -v ${BUNDLER_VERSION}
#######################################################################
# install packages only needed at build time
FROM base as build_deps
ARG BUILD_PACKAGES="git build-essential libpq-dev wget vim curl gzip xz-utils libsqlite3-dev pkg-config python"
ENV BUILD_PACKAGES ${BUILD_PACKAGES}
RUN --mount=type=cache,id=dev-apt-cache,sharing=locked,target=/var/cache/apt \
--mount=type=cache,id=dev-apt-lib,sharing=locked,target=/var/lib/apt \
apt-get update -qq && \
apt-get install --no-install-recommends -y ${BUILD_PACKAGES} \
&& rm -rf /var/lib/apt/lists /var/cache/apt/archives
#######################################################################
# install gems
FROM build_deps as gems
COPY Gemfile* ./
RUN bundle install && rm -rf vendor/bundle/ruby/*/cache
#######################################################################
# install node modules
FROM build_deps as node_modules
RUN curl https://get.volta.sh | bash
RUN volta install node@${NODE_VERSION} yarn@${YARN_VERSION}
COPY package*json ./
COPY yarn.* ./
RUN yarn install
#######################################################################
# install deployment packages
Separately, can I get you to sketch out how you created this app and/or point to a website with the instructions you followed? In the documentation I am working on, I would like to include a Rails API app. I assume that it is something along the lines of:
rails new demo --api
cd demo
npx create-react-app client
If you can correct or confirm this, and add any other considerations, I’ll first document what it takes to get such working, and then look at improving flyctl launch to produce a Dockerfile that requires less customization for future developers coming to fly.io.
So I modified my Dockerfile as you said and unfortunately I don’t think it worked to install node. No evidence of node on the server… Please let me know if you have any other suggestions.
This app was originally made to work on Heroku, but I wanted to try another service to host it. I’m happy to email you (or direct message) details or the github repo if you like. I want to pay back your assistance if I can! Thanks.
#14 25.75 Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
#14 25.75
#14 25.75 current directory:
#14 25.75 /app/vendor/bundle/ruby/2.7.0/gems/nokogiri-1.8.5/ext/nokogiri
#14 25.75 /usr/lib/fullstaq-ruby/versions/2.7.1-jemalloc/bin/ruby -I
#14 25.75 /usr/lib/fullstaq-ruby/versions/2.7.1-jemalloc/lib/ruby/site_ruby/2.7.0
#14 25.75 extconf.rb
#14 25.75 checking if the C compiler accepts ... yes
#14 25.75 Building nokogiri using packaged libraries.
#14 25.75 Using mini_portile version 2.3.0
#14 25.75 checking for gzdopen() in -lz... no
#14 25.75 zlib is missing; necessary for building libxml2
#14 25.75 *** extconf.rb failed ***
#14 25.75 Could not create Makefile due to some reason, probably lack of necessary
#14 25.75 libraries and/or headers. Check the mkmf.log file for more details. You may
#14 25.75 need configuration options.
#14 25.75