Rails app cannot deploy: cannot open shared object file. libc.musl-x86_64.so.1

somazx · February 13, 2023, 2:49pm

Please note: This is a legacy Rails project Ruby 2.7.7, Rails v5.2.4.6

I’ve been grappling with this for a couple hours. It was happening earlier during the bundler process and it seemed like I was able to work past that by modifying Dockerfile:

RUN apt-get update -qq && \
  apt-get install --no-install-recommends -y build-essential libpq-dev libvips node-gyp pkg-config python-is-python3 redis musl-dev
RUN ln -s /usr/lib/x86_64-linux-musl/libc.so /lib/libc.musl-x86_64.so.1

However, now I’m getting the same error at application start:

2023-02-13T14:38:38Z   [info]/usr/local/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:37:in `require': libc.musl-x86_64.so.1: cannot open shared object file: No such file or directory - /rails/vendor/bundle/ruby/2.7.0/gems/msgpack-1.6.0/lib/msgpack/msgpack.so (LoadError)

Full error log:

Recent Events
TIMESTAMP               TYPE            MESSAGE                         
2023-02-13T14:37:16Z    Received        Task received by client        
2023-02-13T14:37:16Z    Task Setup      Building Task Directory        
2023-02-13T14:38:17Z    Started         Task started by client         
2023-02-13T14:38:21Z    Terminated      Exit Code: 1                   
2023-02-13T14:38:21Z    Restarting      Task restarting in 1.158539694s
2023-02-13T14:38:26Z    Started         Task started by client         
2023-02-13T14:38:30Z    Terminated      Exit Code: 1                   
2023-02-13T14:38:30Z    Restarting      Task restarting in 1.128677243s
2023-02-13T14:38:36Z    Started         Task started by client         

2023-02-13T14:38:38Z   [info]   from /rails/vendor/bundle/ruby/2.7.0/gems/bootsnap-1.16.0/lib/bootsnap/load_path_cache.rb:68:in `<top (required)>'
2023-02-13T14:38:38Z   [info]   from /rails/vendor/bundle/ruby/2.7.0/gems/bootsnap-1.16.0/lib/bootsnap.rb:5:in `require_relative'
2023-02-13T14:38:38Z   [info]   from /rails/vendor/bundle/ruby/2.7.0/gems/bootsnap-1.16.0/lib/bootsnap/setup.rb:3:in `require_rel
2023-02-13T14:38:38Z   [info]   from /rails/vendor/bundle/ruby/2.7.0/gems/bootsnap-1.16.0/lib/bootsnap/setup.rb:3:in `require_rel
tive'
2023-02-13T14:38:38Z   [info]   from /rails/vendor/bundle/ruby/2.7.0/gems/bootsnap-1.16.0/lib/bootsnap/setup.rb:3:in `<top (required)>'
2023-02-13T14:38:38Z   [info]   from /usr/local/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:37:in `require'
2023-02-13T14:38:38Z   [info]   from /usr/local/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:37:in `require'
2023-02-13T14:38:38Z   [info]   from /rails/config/boot.rb:4:in `<top (required)>'
2023-02-13T14:38:38Z   [info]   from /rails/bin/rails:3:in `require_relative'
2023-02-13T14:38:38Z   [info]   from /rails/bin/rails:3:in `<main>'
2023-02-13T14:38:38Z   [info]/usr/local/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:37:in `require': libc.musl-x86_64.so.1: cannot open shared object file: No such file or directory - /rails/vendor/bundle/ruby/2.7.0/gems/msgpack-1.6.0/lib/msgpack/msgpack.so (LoadError)
2023-02-13T14:38:38Z   [info]   from /usr/local/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:37:in `require'
2023-02-13T14:38:38Z   [info]   from /rails/vendor/bundle/ruby/2.7.0/gems/msgpack-1.6.0/lib/msgpack.rb:7:in `<top (required)>'
2023-02-13T14:38:38Z   [info]   from /usr/local/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:37:in `require'
2023-02-13T14:38:38Z   [info]   from /usr/local/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:37:in `require'
2023-02-13T14:38:38Z   [info]   from /rails/vendor/bundle/ruby/2.7.0/gems/bootsnap-1.16.0/lib/bootsnap/load_path_cache/store.rb:5:in `block in <top (required)>'
2023-02-13T14:38:38Z   [info]   from /rails/vendor/bundle/ruby/2.7.0/gems/bootsnap-1.16.0/lib/bootsnap/explicit_require.rb:42:in `with_gems'
2023-02-13T14:38:38Z   [info]   from /rails/vendor/bundle/ruby/2.7.0/gems/bootsnap-1.16.0/lib/bootsnap/load_path_cache/store.rb:5:in `<top (required)>'
2023-02-13T14:38:38Z   [info]   from /rails/vendor/bundle/ruby/2.7.0/gems/bootsnap-1.16.0/lib/bootsnap/load_path_cache.rb:68:in `require_relative'
2023-02-13T14:38:38Z   [info]   from /rails/vendor/bundle/ruby/2.7.0/gems/bootsnap-1.16.0/lib/bootsnap/load_path_cache.rb:68:in `<top (required)>'
2023-02-13T14:38:38Z   [info]   from /rails/vendor/bundle/ruby/2.7.0/gems/bootsnap-1.16.0/lib/bootsnap.rb:5:in `require_relative'
2023-02-13T14:38:38Z   [info]   from /rails/vendor/bundle/ruby/2.7.0/gems/bootsnap-1.16.0/lib/bootsnap.rb:5:in `<top (required)>'
2023-02-13T14:38:38Z   [info]   from /rails/vendor/bundle/ruby/2.7.0/gems/bootsnap-1.16.0/lib/bootsnap/setup.rb:3:in `require_relative'
2023-02-13T14:38:38Z   [info]   from /rails/vendor/bundle/ruby/2.7.0/gems/bootsnap-1.16.0/lib/bootsnap/setup.rb:3:in `<top (required)>'
2023-02-13T14:38:38Z   [info]   from /usr/local/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:37:in `require'
2023-02-13T14:38:38Z   [info]   from /usr/local/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:37:in `require'
2023-02-13T14:38:38Z   [info]   from /rails/config/boot.rb:4:in `<top (required)>'
2023-02-13T14:38:38Z   [info]   from /rails/bin/rails:3:in `require_relative'
2023-02-13T14:38:38Z   [info]   from /rails/bin/rails:3:in `<main>'
2023-02-13T14:38:39Z   [info]Starting clean up.
--> v0 failed - Failed due to unhealthy allocations - no stable job version to auto revert to and deploying as v1 

--> Troubleshooting guide at https://fly.io/docs/getting-started/troubleshooting/
Error abort

Thanks for your help!

rubys · February 13, 2023, 3:22pm

Look carefully at the FROM lines in your Dockerfile, and the comments that immediately preceed them. Assuming that you let fly launch generate your Dockerfile in the past two weeks, your Dockerfile should look something like this:

# Make sure RUBY_VERSION matches the Ruby version in .ruby-version and Gemfile
ARG RUBY_VERSION=2.7.7
FROM ruby:$RUBY_VERSION-slim as base

...

# Throw-away build stage to reduce size of final image
FROM base as build

...

# Final stage for app image
FROM base

If the install and the ln command are in the throwaway build stage, then those packages are only available during the build process, but not at deploy. If you move those instructions to the base, they are available to both the build and deploy. If you move them to the final stage they are only available at deploy.

Most debian packages, musl-dev included, have both a -dev and a non-dev version, where the -dev version has additional files needed only during development, and the non-dev version is smaller making deployments faster. So typically you would install the -dev version in the build stage and the non-dev in the final stage. If you are not worried about it, install the -dev version in the base stage.

Now an offer: if you can tell me how to reproduce this, I will change GitHub - rubys/dockerfile-rails: Provide Rails generators to produce Dockerfiles and related files. to detect this and automatically add the correct libraries at the appropriate points. In most cases, all I need to know is which gem or npm module depends on musl. If you are not sure, post the contents of your Gemfile and package.json and I will take it from there.

Fixing dockerfile-rails means that future users of this library will “just work”, and that you will be able to regenerate your dockerfile as you upgrade this application without having to merge in manual changes.

somazx · February 13, 2023, 4:59pm

Hi Same,

Awesome! Thankyou. I missed that there was the the throw-away build section.

Mostly it was the msgpack gem complaining - which appears to be a dependency of bootsnap. Gemfile.lock:

bootsnap (1.16.0)
  msgpack (~> 1.2)

package.json

{
  "name": "inspinia",
  "private": true,
  "dependencies": {
    "datatables.net": "^1.10.22",
    "jquery-mask-plugin": "^1.14.16"
  },
  "packageManager": "yarn@1.22.19"
}

I’d be happy to help improve dockerfile-rails in any way I can. I can email you the Gemfile & Gemfile.lock, if you need them.

rubys · February 13, 2023, 5:29pm

Yes, pleas do email both of those files, as well as your Dockerfile to rubys@fly.io. I’m able to deploy a new project with something approximating your configuration without problems:

And my Gemfile.lock seems to match yours::

    bootsnap (1.16.0)
      msgpack (~> 1.2)

I’m curious as to what is different.

Topic		Replies	Views
Jemalloc error on new app Questions / Help rails	7	1224	July 10, 2023
Rails 7.1 app build fails on bootsnap or assets:precompile Build debugging rails	4	2583	November 14, 2023
Rails 6 app build fails when run from GitHub codespace: LoadError: libffi.so.7: cannot open shared object file Build debugging rails	3	549	April 18, 2023
LoadError in ActiveStorage::Representations::RedirectController#show Questions / Help rails	1	917	November 10, 2022
Error when deploying rails app Questions / Help rails	3	569	October 30, 2022

Rails app cannot deploy: cannot open shared object file. libc.musl-x86_64.so.1

package.json

Related topics