Lion - Memory Usage Problems

John Kitchen wrote:

For a really full discussion go to this link.

FWIW, that link is a PDF version of the same developer documentation I linked to in an earlier post. It is the same link found at the bottom of http://support.apple.com/kb/ht1342, the KBase article mightymilk mentioned in his recent post.

Some things from that doc to consider (somewhat simplified & condensed):

The kernel controls moving pages among the active, inactive, & free lists as needed. It does this based on a threshold determined by the size of physical memory. (See page 12 of the PDF.) Among other things, this means that adding more physical memory (RAM) will not necessarily increase the amount of free memory.

Moving memory pages to the free list is a multi-step process. They must first be moved to the inactive list, & then paged out to disk. However, if the data they contain has not been modified, they can simply be "stolen"-- the pages are immediately moved to the free list without the need to page anything out.

Pages in RAM are often moved between the active & inactive lists. If a process needs data from an inactive page, a soft fault occurs & that page is simple moved back to the active list. A hard fault occurs when the data has been paged out & is no longer in RAM. Hard faults affect performance because the data has to be paged back into RAM from the disk again. It is a very slow process compared to moving pages in RAM, which really just changes which list they are in without touching the data itself.

For this reason, the kernel tries to retain data in inactive memory. It does not move pages to the free list unnecessarily. If it did, hard faults would occur much more frequently, regardless of how much RAM is installed.

Note that Activity Monitor won't tell you much about this. There could be dozens of soft faults occurring between each of its samples of memory use, making it seem like the contents of inactive memory is not changing when in fact it is. As mentioned many times, the best indicator you will get about memory needs from Activity Monitor is the page out numbers. If they are low, the system is not running out of RAM, even if there is very little free memory.

Thanks John.

As long as hardware is cheaper than software dev, we will continue to see things get blotted. That said it's better than my 25+ year experience with Windows

Thanks to you and John. I've wonder what the definitions were

Oh, one thing I have discovered. Chrome is a much "skinnier" browser (memory wise) than either Safari or Firefox. Though, I haven't experience any real system performance difference when I use one over the other in the mix of my other daily apps.

Page Filing, Paging, or PFing is just a Windows terms for what OS X calls Page In and Page Out. The two processes are very similar in nature.

They are similar only to the extent they both involve moving data between RAM & disk storage. Data must always be moved into RAM (paged in) to be used. It may or may not need to be moved back (paged out) to disk. This is true for all modern OS's, although the details of how & when page outs are handled may differ significantly from OS to OS.

However, the basic goal is the same for all performance-oriented OS's: avoid unnecessary paging between RAM & disk because it takes so long to do, relatively speaking. This usually means keeping paged in data in RAM until there is no longer a need for it.

As long as hardware is cheaper than software dev, we will continue to see things get blotted.

If you mean RAM use, the determining factor is how fast data can be moved between RAM & non-volatile storage like HD's or SSD's. The slower that is, the greater the performance penalty for not keeping everything in RAM that you can.

Apps could be designed to have tiny RAM footprints, but they would be insufferably slow.

That won't change until & unless the hardware can move data from storage devices into RAM as quickly as the system can use it. Given that systems run on multicore CPU's & GPU's that can process data many times faster than even the fastest hardware interfaces can support, that isn't going to happen in the foreseeable future.

ColoradoiPodUser wrote:

Yet if I add up the running process' "Real Memory" it never seems to total with what it seems to indicate is being used. Again never saw anything like this under Snow Leopard

You should have seen the same thing with Snow Leopard or Leopard. "Real memory" shows you how much RAM the process is using, but not how much of it is wired (can't be paged out), active (currently in use), or inactive (not currently in use). In addition, some of this memory may be shared with other processes & some of it is private (reserved for that process' use alone).

Activity Monitor doesn't show you how this is distributed among these different categories of RAM. The pie chart & numbers related to it just tell you the totals for all processes. Moreover, it changes much faster than Activity Monitor can track.

So basically the column totals won't add up to the numbers at the bottom because they aren't the total of the same things.

This has not changed in Lion.

I'm not completely against holding data in memory that require quick access. But from what I've seen in Lion, it seems like Inactive RAM isn't being released to programs that need it effeciently or possibly at all...

Activity Monitor can't tell you anything about what specific data is being held in inactive memory, or why it is. This is equally true for any version of OS X to date, not just Lion.

If anything, the trend over the last few OS versions is toward processes that allow the kernel to better determine what can be reused & thus kept in RAM longer to improve performance. I suspect users sometimes mistake this for memory management problems, especially for an app hogging "too much" memory.

It really does take special tools & techniques to determine if this is really true. In fact, without access to the source code, it is very difficult even to use the developer tools to (for example) decide where to insert break points to sample memory use in an app suspected of poor use of memory resources.

Of course, if page outs are going crazy or the system is reduced to snail-speed something is clearly wrong with it, but determining what is causing that usually isn't as simple as we would like it to be.

mightymilk wrote:

I can tell you in all honesty (if you choose to believe my claims), that everytime I notice performance issues I open Activity Monitor... all Free RAM is gone leaving only Inactive, Wired, Active. In no other circumstance is Free RAM completely consumed. I can also tell you there is heavy Page Out/Swap activity.

I do believe you. But do keep in mind that I often see almost no free memory available in both Snow Leopard & Lion, even when doing nothing resource-intensive. The difference is I don't see any appreciable performance issues or spikes in page outs when that happens.

You obviously have a problem that I don't. I don't think the problem is intrinsic to the OS or Safari or Mail -- if it was I think it would affect everyone, or at least everyone with a specific model or configuration or whatever. But since the reports vary so much, I think we are seeing a number of different problems, each with its own cause.

I also have had problems with OS X Lion. On my iMac (a mid 2010 3.06GHz dual core 2 duo with 4 Gb of RAM) the computer just freezes and a hard reset does not resolve the problem; I get the start up tone and then the loading disk spins in perpetuity. I restored the iMac, assuming that there had just been a 'glitch' and within the space of less than 24 hours, I had to do it again. Having subsequently searched the web I noted that I am not alone and worryingly a YouTube post shows a far higher spec machine than mine (newer and with 16Gb of RAM) running alarmingly slowly. I have now restored my Snow Leopard installation as Lion is currently unworkable for me and will not install the upgrade that I have already paid for until Apple produce a suitable patch to resolve the issues that appear to be fairly widespread. I am suprised that Apple appear to be Beta testing their latest operating system in the market place - I moved away from the PC market because I was fed up with problem after problem as immature operating systems were marketed as polished product. Whilst I appreciate that no software product will ever be perfect out of the box, Apple have less scope to get it wrong than the PC market because they control their product specification. How have they got it so badly wrong this time?