NextJS app ran out of memory and crashed but metrics show memory usage not exceeded

Hi, I set up a next.js app to receive an hourly cron job to fetch data from an external api and save it to a separately hosted Postgres DB. I received an email that says my app crashed because it ran out of memory with the following raw log message:

Out of memory: Killed process 532 (node) total-vm:10979168kB, anon-rss:177364kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:3156kB oom_score_adj:0

However when I checked the metrics during that time period it doesn’t seem to exceed the limit so not sure if a memory leak has occurred.

Memory_Usage_2

Any advice on how to debug this issue would be much appreciated!

If this happens rapidly, it might not show up in our metrics.

Looks like there was 1 spike at 5:00 and then a much smaller one at 6:00 (which correlates with the OOM log).

From looking at the first spike, I think this might be what happened: a sudden burst of memory usage.

Do you have a cron running every hour?

Yes, once every hour to fetch about 300-2000 entries of data from an external api.
Not sure why the memory burst would be that high since the data fetched from the api each time is less than 1 mb (probably in the kbs).

I’d try raising the memory on the VM to 512 or 1024 to observe the next spike, see how far it goes without being OOM killed.

You could also enable swap, as outlined here: Swap memory - #2 by OldhamMade (if you don’t want to scale up your memory).

The same data (copied) can occupy different places at the same time, in memory (both kernel and user space). Your app’s structure will also use some space in memory, possibly more than the data itself.

These OOM kills are out of our control unfortunately. We try to use as less as possible from the init that we run in each VM (it shouldn’t be more than a few MBs).

Got it. I’ll review the code as well as I suspect creating a new ApolloClient or batch running the fetch operations using Promise.all() might have something to do with the memory spike.

Thanks for your help!