Hey everyone! I run a local volunteer service for Hudson County, NJ. Fly.io is hosting the security service that handles requests and makes requests to my database. The last successful migration I was able to run was February 29th, 2024 when I added some stuff for groups.
Not sure if things have changed with Fly.io since then, but I tried running a database migration on my service this week and kept running into “out of memory” errors. I think I’m on the Legacy Hobby plan since I joined sometime ~2 years ago, but I don’t think my memory ceiling changed… not sure how to fix this. Whenever I run the migration command, memory utilization spikes from 160 MiB to ~212 MiB before hitting the 217 MiB ceiling and crashing, failing, and restarting itself.
Here’s the details:
2024-06-20T13:09:32.105 app[3d8d9995a29268] ewr [info] [28299.850720] Out of memory: Killed process 314 (node) total-vm:377056kB, anon-rss:97316kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:3484kB oom_score_adj:0
2024-06-20T13:09:32.272 app[3d8d9995a29268] ewr [info] INFO Main child exited with signal (with signal 'SIGKILL', core dumped? false)
2024-06-20T13:09:32.274 app[3d8d9995a29268] ewr [info] INFO Process appears to have been OOM killed!
2024-06-20T13:09:32.275 app[3d8d9995a29268] ewr [info] INFO Starting clean up.
2024-06-20T13:09:32.276 app[3d8d9995a29268] ewr [info] WARN could not unmount /rootfs: EINVAL: Invalid argument
2024-06-20T13:09:32.277 app[3d8d9995a29268] ewr [info] [28300.023379] reboot: Restarting system
2024-06-20T13:09:32.370 app[3d8d9995a29268] ewr [info] Out of memory: Killed process
2024-06-20T13:09:32.428 runner[3d8d9995a29268] ewr [info] machine did not have a restart policy, defaulting to restart
2024-06-20T13:09:32.600 app[3d8d9995a29268] ewr [info] 2024-06-20T13:09:32.600605113 [01J0T0G1MHM49J8A1S7BB3WNR7:main] Running Firecracker v1.7.0
2024-06-20T13:09:32.931 app[3d8d9995a29268] ewr [info] [ 0.268426] PCI: Fatal: No config space access function found
2024-06-20T13:09:33.214 app[3d8d9995a29268] ewr [info] INFO Starting init (commit: f7402432)...
2024-06-20T13:09:33.246 app[3d8d9995a29268] ewr [info] INFO Preparing to run: `docker-entrypoint.sh node dist/main` as root
2024-06-20T13:09:33.252 app[3d8d9995a29268] ewr [info] INFO [fly api proxy] listening at /.fly/api
2024-06-20T13:09:33.257 app[3d8d9995a29268] ewr [info] 2024/06/20 13:09:33 INFO SSH listening listen_address=[fdaa:1:d84a:a7b:cc:12f0:2d9a:2]:22 dns_server=[fdaa::3]:53
2024-06-20T13:09:33.268 runner[3d8d9995a29268] ewr [info] Machine started in 738ms
2024-06-20T13:09:37.417 app[3d8d9995a29268] ewr [info] [Nest] 314 - 06/20/2024, 1:09:37 PM LOG [NestFactory] Starting Nest application...
2024-06-20T13:09:37.494 app[3d8d9995a29268] ewr [info] [Nest] 314 - 06/20/2024, 1:09:37 PM LOG [InstanceLoader] AppModule dependencies initialized +120ms`
M