kernel panic when deploying app

Questions / Help Build debugging
1

Hi, i tried to deploy a phoenix app (1.6.7, Seattle region both app and postgres) and it is returning kernel panic error

...
--> This release will not be available until the release command succeeds.
	 Starting instance
	 Configuring virtual machine
	 Pulling container image
	 Unpacking image
	 Preparing kernel init
	 Configuring firecracker
	 Starting virtual machine
	 Starting init (commit: 252b7bd)...
	 Preparing to run: `/app/bin/migrate` as nobody
	 Error: UnhandledIoError(Os { code: 2, kind: NotFound, message: "No such file or directory" })
	 [    0.097198] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100
	 [    0.098398] CPU: 0 PID: 1 Comm: init Not tainted 5.12.2 #1
	 [    0.099241] Call Trace:
	 [    0.099584]  show_stack+0x52/0x58
	 [    0.100082]  dump_stack+0x6b/0x86
	 [    0.100591]  panic+0xfb/0x2bc
	 [    0.101041]  do_exit.cold+0x60/0xb0
	 [    0.101529]  do_group_exit+0x3b/0xb0
	 [    0.102092]  __x64_sys_exit_group+0x18/0x20
	 [    0.102705]  do_syscall_64+0x38/0x50
	 [    0.103252]  entry_SYSCALL_64_after_hwframe+0x44/0xae
	 [    0.104013] RIP: 0033:0x6fea55
	 [    0.104472] Code: eb ef 48 8b 76 28 e9 76 05 00 00 64 48 8b 04 25 00 00 00 00 48 8b b0 b0 00 00 00 e9 af ff ff ff 48 63 ff b8 e7 00 00 00 0f 05 <ba> 3c 00 00 00 48 89 d0 0f 05 eb f9 66 2e 0f 1f 84 00 00 00 00 00
	 [    0.107014] RSP: 002b:00007ffcf59a51a8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
	 [    0.108049] RAX: ffffffffffffffda RBX: 00000000005d0250 RCX: 00000000006fea55
	 [    0.108775] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000001
	 [    0.110312] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
	 [    0.110312] R10: 0000000000000000 R11: 0000000000000246 R12: 00007ffcf59a5208
	 [    0.111684] Kernel Offset: disabled
: 0000000000000000 R15: 0000000000000000
	 [    0.111684] Kernel Offset: disabled
	 [    0.111973] Rebooting in 1 seconds..
	 Starting instance
	 Configuring virtual machine
	 Pulling container image
	 Unpacking image
	 Preparing kernel init
	 Configuring firecracker
	 Starting virtual machine
	 Starting init (commit: 252b7bd)...
	 Preparing to run: `/app/bin/migrate` as nobody
	 Error: UnhandledIoError(Os { code: 2, kind: NotFound, message: "No such file or directory" })
	 [    0.097198] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100
	 [    0.098398] CPU: 0 PID: 1 Comm: init Not tainted 5.12.2 #1
	 [    0.099241] Call Trace:
	 [    0.099584]  show_stack+0x52/0x58
	 [    0.100082]  dump_stack+0x6b/0x86
	 [    0.100591]  panic+0xfb/0x2bc
	 [    0.101041]  do_exit.cold+0x60/0xb0
	 [    0.101529]  do_group_exit+0x3b/0xb0
	 [    0.102092]  __x64_sys_exit_group+0x18/0x20
	 [    0.102705]  do_syscall_64+0x38/0x50
	 [    0.103252]  entry_SYSCALL_64_after_hwframe+0x44/0xae
	 [    0.104013] RIP: 0033:0x6fea55
	 [    0.104472] Code: eb ef 48 8b 76 28 e9 76 05 00 00 64 48 8b 04 25 00 00 00 00 48 8b b0 b0 00 00 00 e9 af ff ff ff 48 63 ff b8 e7 00 00 00 0f 05 <ba> 3c 00 00 00 48 89 d0 0f 05 eb f9 66 2e 0f 1f 84 00 00 00 00 00
	 [    0.107014] RSP: 002b:00007ffcf59a51a8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
	 [    0.108049] RAX: ffffffffffffffda RBX: 00000000005d0250 RCX: 00000000006fea55
	 [    0.108775] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000001
	 [    0.110312] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
	 [    0.110312] R10: 0000000000000000 R11: 0000000000000246 R12: 00007ffcf59a5208
	 [    0.111684] Kernel Offset: disabled
: 0000000000000000 R15: 0000000000000000
	 [    0.111684] Kernel Offset: disabled
	 [    0.111973] Rebooting in 1 seconds..
Error release command failed, deployment aborted

just a new app generated from phoenix, dockerfile and toml. the initial fly launch went fine, except when i make tiny changes and redeploying it

2 Likes
2 
Preparing to run: `/app/bin/migrate` as nobody
	 Error: UnhandledIoError(Os { code: 2, kind: NotFound, message: "No such file or directory" })
	 [    0.097198] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100

Ah, forgot the mix phx.gen.release --ecto part :man_facepalming:. Thats why it cant find /bin/migrate.

2 Likes
3

Thx to jaeyson. You saved my day after running into the same problem and trying one deploy after another :sweat_smile:.

To give an additional info: until recently if someone creates a new phoenix project and starts fly launch the release files were built as expected. Now, it seems that if you just use fly launch the ecto part release files (rel/overlays/bin/migrate and rel/overlays/bin/migrate.bat) are not built by default and therefore missing.

So, as jaeyson wrote, run mix phx.gen.release --ecto before running fly launch.

3 Likes
4

Phoenix 1.6.7 would not find ecto in some cases, causing phx.gen.release not to include it. Phoenix 1.6.8 has been released which fixes this. Thanks!

3 Likes
5

Perfect Chris! Thanks a lot!