Error connecting to the API on release

Questions / Help
1

I’ve been getting a error all day when trying to deploy a version of my application to fly.io. Everything seems to work up until when the release is about to be created.

==> Verifying app config
--> Verified app config
==> Building image
Searching for image 'tzauth:wb91khgj42k17bdjf2s3lbnrain6dvv1' locally...
image found: sha256:5e5dbcc64481cef126cb47921b36eb6d76d8c2dd570d034326af9d536c63c88c
==> Pushing image to fly
The push refers to repository [registry.fly.io/tzauth-staging]
79145a169272: Layer already exists 
deployment-1660243269: digest: sha256:9908a78ae458d33e3bd1f8233bd68d68097387e442c6ce39be68bf37342957b9 size: 528
--> Pushing image done
==> Creating release
Error You ran into an error connecting to the Fly API. If the error persists, keep this request ID to help Fly Support track it down: 01GA73C7Y16BW3JA8761V6566H-fra
2

on our end, I can see that our API is having trouble with this image, but I don’t know why that is yet. From those logs I’d guess you might be building locally when you deploy. You might be able to workaround this issue by deploying remotely to one of our builders instead. (For newer versions of flyctl that should be the default, but you can explicitly specify this with --remote-only)

what sort of image are you trying to deploy? what options are you passing to fly deploy when you’re deploying?

3

I’m using nix to build the image, the problems started when I changed to use nix2container because I had issues getting the standard way to come up.

I’m running something like this command: flyctl deploy -c /nix/store/7zar931s25gsaimd8c348axqmr98mm7z-fly.toml -i tzauth:pj35i6z1mj6pmb8b69wnjf523a94gj6h --local-only --verbose

I’m currently trying to get something going at all so that’s why I’m deploying from my local machine. If there’s something wrong with the image I’ll try to revert to the standard way of building docker images with nix.

4

Reverted to building the normal nix way instead and now I’m back to this error:
Any ideas how to fix this @eli ?

2022-08-11T19:17:09Z runner[278eb4c7] fra [info]Starting instance
2022-08-11T19:17:09Z runner[278eb4c7] fra [info]Configuring virtual machine
2022-08-11T19:17:09Z runner[278eb4c7] fra [info]Pulling container image
2022-08-11T19:17:14Z runner[278eb4c7] fra [info]Unpacking image
2022-08-11T19:17:16Z runner[278eb4c7] fra [info]Preparing kernel init
2022-08-11T19:17:16Z runner[278eb4c7] fra [info]Configuring firecracker
2022-08-11T19:17:17Z runner[278eb4c7] fra [info]Starting virtual machine
2022-08-11T19:17:17Z app[278eb4c7] fra [info]Starting init (commit: c86b3dc)...
2022-08-11T19:17:17Z app[278eb4c7] fra [info]Preparing to run: `/nix/store/243rwd6xjvldkg36qbg37sznxgdi4sn4-app/bin/app` as root
2022-08-11T19:17:17Z app[278eb4c7] fra [info]Error: UnhandledIoError(Os { code: 2, kind: NotFound, message: "No such file or directory" })
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.139775] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.140556] CPU: 0 PID: 1 Comm: init Not tainted 5.12.2 #1
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.141115] Call Trace:
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.141373]  show_stack+0x52/0x58
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.141741]  dump_stack+0x6b/0x86
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.142088]  panic+0xfb/0x2bc
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.142399]  do_exit.cold+0x60/0xb0
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.142756]  do_group_exit+0x3b/0xb0
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.143132]  __x64_sys_exit_group+0x18/0x20
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.143563]  do_syscall_64+0x38/0x50
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.143919]  entry_SYSCALL_64_after_hwframe+0x44/0xae
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.144443] RIP: 0033:0x6ff9c5
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.144766] Code: eb ef 48 8b 76 28 e9 76 05 00 00 64 48 8b 04 25 00 00 00 00 48 8b b0 b0 00 00 00 e9 af ff ff ff 48 63 ff b8 e7 00 00 00 0f 05 <ba> 3c 00 00 00 48 89 d0 0f 05 eb f9 66 2e 0f 1f 84 00 00 00 00 00
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.146646] RSP: 002b:00007ffc8f75a2c8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.147412] RAX: ffffffffffffffda RBX: 00000000004f0ed0 RCX: 00000000006ff9c5
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.148137] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000001
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.148877] RBP: 0000000000000001 R08: 0000000000000000 R09: 0000000000000000
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.149630] R10: 0000000000000000 R11: 0000000000000246 R12: 00007ffc8f75a328
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.150328] R13: 00007ffc8f75a338 R14: 0000000000000000 R15: 0000000000000000
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.151074] Kernel Offset: disabled
2022-08-11T19:17:17Z app[278eb4c7] fra [info][    0.151434] Rebooting in 1 seconds..
5

Found this post which was the problem for me as well. Would be nice with a better error message :sweat_smile:

6

Glad to see that you found it (and thanks for confirming what the issue was). I’ll see what we can do about that error message. It’s awesome to see so many people using nix to deploy on Fly though :sweat_smile:

7

Yeah, I’ve used GitHub - LutrisEng/nix-fly-template: Build an application with Nix and deploy it on Fly.io as inspiration and it makes it a breeze to do deployments :smile: To make a new deployment (now that I got it working) is as simple as nix run .#deploy-staging