Cumulative memory issue

Questions / Help
1

My FastAPI application is doing memory-intensive tasks only when a request comes in. When the request is done, there’s no more task other than keeping the server running. It’s running on 4GB RAM.

However, the memory usage on fly-metrics seems to be cumulative, it seems to add up. My expected behavior is that after the request is finished, it’ll get back to its previous memory usage level.

Let’s say the memory usage was 1GB before the request arrived. During the request, the memory usage rises to 3GB. So, after the request is finished, it should get back to around 1GB again.

However, what I’m noticing is, it stays at 3GB. When I make another request, the memory usage rises again and stays there.

Here’s the screenshot:

image
image906×463 30.6 KB

.

The marked period is when the request ran and finished. But as you see, the memory usage stayed at the same level even after finishing the request.

It went down after restarting. I ran the app in my local and didn’t see this behavior with local memory. Then why is this happening in production?

I’m so now confused :slight_smile:

Can you please tell me what can go wrong?

2

However, the memory usage on fly-metrics seems to be cumulative, it seems to add up. My expected behavior is that after the request is finished, it’ll get back to its previous memory usage level.

Memory typically won’t be evicted right away. It will generally be preserved in memory as cache, which will help speed up subsequent requests that may need access to that data. Unfortunately, there’s no distinction within the dashboard, which can make things pretty confusing. I’ll make a note about improving this. In any case, cache will be evicted in order to make room for new allocations as your memory utilization continues to grow.

If you ssh into your Machine, you can run free -m to get a better picture of what’s going on. If you are seeing anything different, let me know!

1 Like
3

I’ll also note that FastAPI is running on a JS VM which was designed for a browser, and memory that has been garbage collected may not be immediately returned to the OS; and perhaps may never be returned.

1 Like
4

Thanks for your explanation.

5

Okay, in this case, how can we make it do that?

6

I’m not sure. javascript - How to request the Garbage Collector in node.js to run? - Stack Overflow may help.

7

Thanks for sharing. However, I’m not sure how I may implement that into my FastAPI app.

8

Generally speaking, if the memory you’re referring to is indeed “cache” it’s not something you should be concerned with. It’s pretty common for applications and databases to aggressively cache as a way of improving performance. Any memory living in cache will be evicted/drained when additional memory is required by other requests, processes, etc. and should not lead to OOM related issues.

9

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.