I'm looking for a metric I can use to accurately measure memory usage so I can auto-scale my server application. This is difficult, because malloc doesn't return freed memory to the OS and I'm not sure how to get a metric that represents used and free memory.
My server has high peak memory usage for sustained amounts of time. This is because it is a document-syncing server and functions by loading the document (a whiteboard) into memory. When all clients disconnect, it can then close the document and remove it from memory. You can see an example of a document/whiteboard here.
Out of the box, I can measure the OS's free memory, but this metric has a problem. The problem is that the used memory usage never seems to go down, even when a whiteboard is closed and removed from memory. From what I've learned on other posts (Memory leaking in Vapor app), it seems this is normal behavior for malloc. I understand that when memory is freed malloc might recycle it, but it doesn’t return it to the OS. That means that the process memory reaches a peak and then doesn’t go back down even if there is plenty of freed heap space ready to be reused
Here is a graph of memory usage in my app over a couple months. You can see it goes up and then plateaus at peak memory usage. It never goes down significantly, except when I restart the app (those are the sharp drops downward).
What metric can I use to determine if I need more or less servers at a given moment? For example, is there an API to ask the allocator how much memory is freed and available for recycling? Maybe the metric has something to do with paging (which I know very little about)?
If I use the out-of-the-box system memory metric, then my application will scale out but it will never scale in, because that metric never goes down.
I'm also not sure what happens if my application reaches 100% memory usage. I think swapping happens and it gets slower, but I'm not sure how bad that would be...
If this API is specific to the OS, I am using the official Swift Docker image based on Ubuntu 18.04.