How to read erl_crash.dump

Questions / Help
1

Hey there,

Is there a way for me to read my erl_crash.dump somehow when my app crashes on startup?

There are 0 helpful logs in the logs in fly interface. And when I console in, I have 2 seconds to find and read it. (I can’t even find it, but I think it’s because it’s not persisted)

Would love any help on this.

Here’s my current logs

 2024-06-18T09:01:51.235 app[2874d34a034548] sjc [info] 2024-06-18T09:01:51.235224484 [01J0N998244PB9TK6R7H38YX97:main] Running Firecracker v1.7.0

2024-06-18T09:01:51.662 app[2874d34a034548] sjc [info] INFO Preparing to run: `/app/bin/server` as nobody

2024-06-18T09:01:51.667 app[2874d34a034548] sjc [info] INFO [fly api proxy] listening at /.fly/api

2024-06-18T09:01:51.680 runner[2874d34a034548] sjc [info] Machine created and started in 9.843s

2024-06-18T09:01:55.407 health[2874d34a034548] sjc [error] Health check on port 8080 has failed. Your app is not responding properly. Services exposed on ports [80, 443] will have intermittent failures until the health check passes.

2024-06-18T09:01:56.529 app[2874d34a034548] sjc [info] Runtime terminating during boot (terminating)

2024-06-18T09:01:56.674 app[2874d34a034548] sjc [info] WARN Reaped child process with pid: 377 and signal: SIGUSR1, core dumped? false

2024-06-18T09:01:56.846 app[2874d34a034548] sjc [info] Crash dump is being written to: erl_crash.dump...done

2024-06-18T09:01:57.676 app[2874d34a034548] sjc [info] INFO Main child exited normally with code: 1

2024-06-18T09:01:57.690 app[2874d34a034548] sjc [info] INFO Starting clean up.

2024-06-18T09:01:57.692 app[2874d34a034548] sjc [info] WARN could not unmount /rootfs: EINVAL: Invalid argument

2024-06-18T09:01:57.693 app[2874d34a034548] sjc [info] [ 6.384490] reboot: Restarting system

2024-06-18T09:02:01.974 runner[2874d34a034548] sjc [info] machine did not have a restart policy, defaulting to restart

2024-06-18T09:02:02.141 app[2874d34a034548] sjc [info] 2024-06-18T09:02:02.141526065 [01J0N998244PB9TK6R7H38YX97:main] Running Firecracker v1.7.0
1 Like
2

I’ve had to debug a bug previously when the machine exited with no elixir logs. I’ve changed my Dockerfile to:

- CMD ["/app/bin/server"]
+ CMD ["sleep", "infinity"]

Then you can deploy once more and fly ssh console. Do note your app will not be online during that time.

This post has more debugging tips:

1 Like
3

As an alternative, you can get the crash dump to output to stderr by adding the following exports to rel/env.sh.eex:

export ERL_CRASH_DUMP=/dev/stderr
export ERL_CRASH_DUMP_BYTES=8192

This will give you the output in fly logs or in our web UI.

Hope this helps!

5 Likes
4

This did the trick!
Thank you both so much!

1 Like
5

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.