Elixir/Phoenix startup issues, managed Postgres database

Phoenix
, ,
1

Launched a phoenix app with a managed postgres cluster. Got a 503 error. Saw in live logs that the DATABASE_URL was unset and a crash stack. So I did:

fly secrets set DATABASE_URL="ecto://pgdb-1zvn90kjddprkpew:asdfasdf@direct.1zvn90kjddprkpew.flympg.net/pgdb-1zvn90kjddprkpew?sslmode=require" --stage

fly deploy

now when I look at the log of the machine, it appears to just hang .. any suggestions?

regards, Michael

➜  tv git:(deploy-fly-io) ✗ fly logs -a tyndale-app -i 683d551b151e58
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]size: 16384
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]used: 10731
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]objs: 16965
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]depth: 7
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]=hash_table:node_table
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]size: 16
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]used: 1
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]objs: 1
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]depth: 1
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]=hash_table:dist_table
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]size: 16
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]used: 2
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]objs: 2
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]depth: 1
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]=allocated_areas
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]sys_misc: 57816
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]static: 1053984
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]atom_space: 1311040 1290307
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]atom_table: 1106057
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]module_table: 2148984
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]export_table: 2015652
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]export_list: 10146800
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]register_table: 372
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]fun_table: 131186
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]module_refs: 203088
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]loaded_code: 62210968
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]dist_table: 1043
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]node_table: 291
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]bits_bufs_size: 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]bif_timer: 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]process_table: 3145728
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]port_table: 786432
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]ets_misc: 131072
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]external_alloc: 62210968
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]=allocator:sys_alloc
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option e: true
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option m: libc
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option tt: 131072
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option tp: 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]=allocator:temp_alloc[0]
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]versions: 2.1 3.0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option e: true
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option t: 5
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option ramv: false
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option atags: false
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option sbct: 524288
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option asbcst: 4145152
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option rsbcst: 90
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option rsbcmt: 80
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option rmbcmt: 100
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option mmbcs: 131072
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option mmsbc: 256
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option mmmbc: 18446744073709551615
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option lmbcs: 10485760
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option smbcs: 1048576
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option mbcgs: 10
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option acul: 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option acful: 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option acnl: 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option acfml: 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option cp: undefined
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option mbsd: 3
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]option as: gf
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]mbcs blocks[sys_alloc] count: 0 0 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]mbcs blocks[sys_alloc] size: 0 0 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]mbcs blocks[temp_alloc] count: 0 5 5
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]mbcs blocks[temp_alloc] size: 0 11288 11288
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]mbcs blocks[sl_alloc] count: 0 0 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]mbcs blocks[sl_alloc] size: 0 0 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]mbcs blocks[std_alloc] count: 0 0 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]mbcs blocks[std_alloc] size: 0 0 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]mbcs blocks[ll_alloc] count: 0 0 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]mbcs blocks[ll_alloc] size: 0 0 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]mbcs blocks[eheap_alloc] count: 0 0 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]mbcs blocks[eheap_alloc] size: 0 0 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]mbcs blocks[ets_alloc] count: 0 0 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]mbcs blocks[ets_alloc] size: 0 0 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]mbcs blocks[fix_alloc] count: 0 0 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]mbcs blocks[fix_alloc] size: 0 0 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info]mbcs blocks[literal_alloc] count: 0 0 0
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info] INFO Main child exited normally with code: 1
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info] WARN Reaped child process with pid: 729 and signal: SIGUSR1, core dumped? false
2025-06-30T18:36:04Z app[683d551b151e58] yyz [info] INFO Starting clean up.
2025-06-30T18:36:05Z app[683d551b151e58] yyz [info] WARN could not unmount /rootfs: EINVAL: Invalid argument
2025-06-30T18:36:05Z app[683d551b151e58] yyz [info][   12.411506] reboot: Restarting system
2025-06-30T18:36:05Z runner[683d551b151e58] yyz [info]machine has reached its max restart count of 10
2025-06-30T18:36:08Z proxy[683d551b151e58] yyz [error][PR04] could not find a good candidate within 20 attempts at load balancing
2025-06-30T18:36:08Z proxy[683d551b151e58] yyz [error][PC05] timed out while connecting to your instance. this indicates a problem with your app (hint: look at your logs and metrics)
2025-06-30T18:37:01Z runner[683d551b151e58] yyz [info]Pulling container image registry.fly.io/tyndale-app@sha256:871ec5f98e500219d750bb5f6c97e3525f4d4de8c9901d6341269cc253ddcb2c
2025-06-30T18:37:01Z runner[683d551b151e58] yyz [info]Container image registry.fly.io/tyndale-app@sha256:871ec5f98e500219d750bb5f6c97e3525f4d4de8c9901d6341269cc253ddcb2c already prepared
2025-06-30T18:37:01Z runner[683d551b151e58] yyz [info]Configuring firecracker
2025-06-30T18:43:42Z runner[683d551b151e58] yyz [info]Pulling container image registry.fly.io/tyndale-app@sha256:871ec5f98e500219d750bb5f6c97e3525f4d4de8c9901d6341269cc253ddcb2c
2025-06-30T18:43:42Z runner[683d551b151e58] yyz [info]Container image registry.fly.io/tyndale-app@sha256:871ec5f98e500219d750bb5f6c97e3525f4d4de8c9901d6341269cc253ddcb2c already prepared
2025-06-30T18:43:43Z runner[683d551b151e58] yyz [info]Configuring firecracker
2025-06-30T18:44:52Z runner[683d551b151e58] yyz [info]Pulling container image registry.fly.io/tyndale-app@sha256:871ec5f98e500219d750bb5f6c97e3525f4d4de8c9901d6341269cc253ddcb2c
2025-06-30T18:44:52Z runner[683d551b151e58] yyz [info]Container image registry.fly.io/tyndale-app@sha256:871ec5f98e500219d750bb5f6c97e3525f4d4de8c9901d6341269cc253ddcb2c already prepared
2025-06-30T18:44:52Z runner[683d551b151e58] yyz [info]Configuring firecracker
2025-06-30T18:46:08Z runner[683d551b151e58] yyz [info]Pulling container image registry.fly.io/tyndale-app@sha256:871ec5f98e500219d750bb5f6c97e3525f4d4de8c9901d6341269cc253ddcb2c
2025-06-30T18:46:08Z runner[683d551b151e58] yyz [info]Container image registry.fly.io/tyndale-app@sha256:871ec5f98e500219d750bb5f6c97e3525f4d4de8c9901d6341269cc253ddcb2c already prepared
2025-06-30T18:46:08Z runner[683d551b151e58] yyz [info]Configuring firecracker
3

Hm… I would try destroying this Machine (fly m destroy 683d551b151e58) and then redeploying. On top of that, it might be prudent to start fly logs in a separate terminal beforehand—and leave it running the whole time—so you have a better chance of seeing the full set.

There really should have been more stuff, like Preparing to run /app/bin/tyndale as root, in the logs…

Hope this helps a little!

4 
WARN Reaped child process with pid: 729 and signal: SIGUSR1, core dumped? false

This doesn’t sound good. How much RAM are you allocating to the machine? I am guessing that your main process has been killed by the process manager for consuming too much memory. I’d try launching with more RAM to start off with, and then if that turns out to be the problem, start optimising your app’s memory requirements.

5

Yeah, that was the problem. I chose Development 1GB and ran out of memory. I wish there was a single-node 2GB or 4GB.

6

There’s a single shared CPU 2GB, see here: Fly.io Resource Pricing · Fly Docs

1 Like
7

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.