release_command failed - beam.smp: __extendhfdf2: symbol not found

My app suddenly stopped running without warning (it is in “suspended” state).
I have not touched this app in a year so it does not come from recent changes.

I get the following error when I try to deploy:

-------
 ✖ release_command failed
-------
Error release_command failed running on machine e82d37dc2517e8 with exit code 127.
Check its logs: here's the last 100 lines below, or run 'fly logs -i e82d37dc2517e8':
  Pulling container image registry.fly.io/nara:deployment-01HE1916F82RVVPJRDT8H24HBY
  Successfully prepared image registry.fly.io/nara:deployment-01HE1916F82RVVPJRDT8H24HBY (28.116207788s)
  Configuring firecracker
  [    0.053063] PCI: Fatal: No config space access function found
   INFO Starting init (commit: 15238e9)...
   INFO Preparing to run: `/app/bin/migrate` as nobody
   INFO [fly api proxy] listening at /.fly/api
  2023/10/30 21:57:00 listening on [fdaa:0:357c:a7b:16a:8c96:386e:2]:22 (DNS: [fdaa::3]:53)
  Error relocating /app/erts-12.3.2.13/bin/beam.smp: __extendhfdf2: symbol not found
  Error relocating /app/erts-12.3.2.13/bin/beam.smp: __truncdfhf2: symbol not found
  Error relocating /app/erts-12.3.2.13/bin/beam.smp: __truncxfhf2: symbol not found
   INFO Main child exited normally with code: 127
   INFO Starting clean up.
   WARN hallpass exited, pid: 307, status: signal: 15 (SIGTERM)
  2023/10/30 21:57:01 listening on [fdaa:0:357c:a7b:16a:8c96:386e:2]:22 (DNS: [fdaa::3]:53)
  [    2.358113] reboot: Restarting system
  machine restart policy set to 'no', not restarting
-------
Error: release command failed - aborting deployment. error release_command machine e82d37dc2517e8 exited with non-zero status of 127

This is happening to me again too. My release is hanging randomly.

I have had a chat with Fly’s paid support team and was able to resolve my issue. Here are the suggestions they made so everyone can benefit from it. The second message is the one that worked for me.

TL;DR: I updated to a recent alpine runtime image in my docker file and had to update a dependency (ecto in my case)

1st msg:

There are a few things to look at when getting an error like this:

  1. Review Dependencies: Verify that all the dependencies required by your app are correctly installed and configured in the Docker image. Missing dependencies could cause the missing symbols error.

  2. Inspect the Dockerfile: Look at your Dockerfile used to build the image to ensure it’s set up to include all necessary libraries and binaries that your application depends on.

  3. Review the Release Command: The release_command that failed (/app/bin/migrate ) may have specific requirements or environment variables that are not correctly set. Check the script or command for any paths or environment variables that need to be defined.

  • Typically this error occurs when you need to update the secret or envs to your application because of a library or dependency being out of date.
  • Also do you have auto-stop set for your application? If you do then you won’t get any notifications since it’s working as configured.

2nd msg:

This is a shot in the dark but are you using the Alpine Linux base image? If so, you might need to bump your alpine version as described in this post: https://stackoverflow.com/a/72613731.

1 Like