Lion - Memory Usage Problems

Lads! add me to the list too for this issue. Spent this arvo reading the 48 pages of this thread.. interesting for the most (excluding the kid squabbling). I'll add my two bobs worths as:

• notice this consistent/ persistent memory creep of the ALLOCATED UNUSED portion on the blue area of the activity monitor.app.. in some cases 100MB/sec when particular some apps run
• first noticed this at 10.7.1....
• of these apps over long duration, my entire 32GB of real memory (ram) is evenutally consumed as 80-90% of ALLOCATED UNUSED (blue) in the system memory display
• agree with many of you including Mr John Kitchen (STK/EMC???) that the monitor but not be accurate. However when I see it inicating continually the incremental amount of ALLOCATED UNUSED (blue) this makes me have a look
• I noticed that this happens when I am utilising NON-APPLE software... such archiving software (LTO4 Ultrium), media asset manager and a few others however other than the subjective observation that I see this when they are running, I cant tie them to the cause of ALLOCATED UNUSED (blue) memory
• I dont see this ever when I use FCPX, Motion, Aperture, Nuke blah blah..... strange .. only when some 3rd party ISV is running. (I have contacted one ISV about this).. so this is a bit too subjective
• shared memory for all tasks/apps is "low" meaning no large numbers
• Like most of you I end up running our of real and my MAC PRO read and writes pages like an old monk! My system was grinding to a halt and I would need to restart (IPL) 🙂 )
• would appreciate any guidance/advice on how to diagnose this.

My work around is the use of the unix "purge" command on 10.7.3.. works ok but is not the answer as you would agree.

So..... I'll be watching this thread for any info.

Love the thread BTW!

warwick

Hong Kong

An image of Activity Monitor would have helped, but anyway..

Warwick Teale wrote:

of these apps over long duration, my entire 32GB of real memory (ram) is evenutally consumed as 80-90% of ALLOCATED UNUSED (blue) in the system memory display

If with "allocated unused" you mean the Inactive memory the choice of term is a bit wrong. My last understanding about Inactive memory is that the pageout daemon splits it into 2 queues:

- The pages there were previously Active (thus used/referenced in the past by the processes)

- The Zero Filled pages. These are the pages that gets allocated (via malloc() for example) by the processes. Once a process make a request for memory, the memory gets allocated as Inactive. Only once the process(es) physically access their content (even just reading it) they get zero filled (for security reasons) and the page gets moved into the Active queue, creating the mapping and everything that is necessary. ZF pages should be between the firsts to go (after the speculative pages actually) once the pageout daemon starts.

What it may happen in your case is that your applications are memory hungry and allocate large chunks of RAM. Which is clearly seldom used since most of it remains inactive. As soon as your free memory gets too low those ZF pages should be freed before hitting the backing storage (which happens freeing the modified inactive pages). If such is the case OS X shouldn't page out.

One thing you can do to test it is to quit the suspected application and see if the Inactive memory gets low. Or to try to start them one by one, having large delay (hours?) between one and another to see which can be the culprit.

It isn't very clear to me yet what "purge" does. I see that it frees speculative, inactive and active pages. Chances that running it just makes the whole system slower for some time are high. For the simple reason that all "caches" are gone.

Thanks Michelasso for the explanation. Yep I get malloc/getmain and auxillary storage/VM stuff and all that. I am assuming correctly that this explanation carries also to some previous OSX's... (y/n).

Thus if that assumtpion is true, then WHY is this behaviour/artefact of this ALLOCATED UNUSED (correctlabel or not), is exaccebated in recent OSX LION 10.7.x maint releases?

Or to put it another way, why was this behaviour exhobiated in pre OSX 10.7 flavours of the OS?

I feel there is a defect in there some where in the recent OS. Like many of us, our suites are used for production work and we know them intimatley 🙂 .. so when they change habit we recognise it quickly. This is the case here.

Simply I noticed this over time when editing with FCPX.. and its not FCPX. lol!

regarding the ACTIVITY MONITOR.app capture.. I'll do this on the weekend when I can restar the machine.

APPS: y... sound advice to cancel the apps and watch the memory. This was what I had done weeks back when I first spotted it. Looks like the allocation is asynchronously allocated... or whatever the term s.. so thatwhat ever service needs a services tha needs memeory (malloc), there is no associated "ownership"..and thus not cleaned up when the orginal requested get killed.. hence what might be exhibted as this leak.

anyway thanks for the detail.. appreciate it mate.

w

HK

I have feeling that 'memory leaking' problem is solved with newest Safari update 5.1.4. Am I right?

no idea 😁 .. the issue that I describe is unrelated to Safari... never had an issue like this with it FWIW.

W

I believe what you observe is memory allocations in the 'buffer space', which is not directly allocated by the application, but instead allocated tto support the application. Some could simplistically call that 'the disk cache' (but it also contains all sorts of other buffers). It is accounted for in the 'inactive memory', because it's contents can be at any time thrown away (and the memory made 'free'), and about the only cost would be to re-read the same data from disk/network again. I believe huge allocations like this are typical for multimedia type processing.. and you seems to do just that with FCPX.

Your other observation, and right so (matches mine) is that typically third party software allocates lots of memory they never use, or in general have suboptimal memory footprint. This is, directly speaking sloppy programming! These people think that because you have an Macintosh with 16 GB RAM, you are supposed to dedicate it all to their "productivity" or "professional" application. They just don't care to read, much less understand the programming guidelines Apple has published (what we discussed just recently in this thread).

Also, it is important to note, that sometime it's not the "application" but rather an application-specific device driver that exibits this improper buffer memory handling, or rather excessive usage. In situations like that, the memory may not be freed when you stop the application, but rather, when you disconnect the hardware or shut down the software emulation layer (Framework) etc.

We can only blame Apple for not properly managing memory for applications that behave (and follow the rules). It seems they have addressed the memory usage of their own software quite a bit in recent updates.

dkalchev wrote:

I believe what you observe is memory allocations in the 'buffer space', which is not directly allocated by the application, but instead allocated tto support the application. Some could simplistically call that 'the disk cache' (but it also contains all sorts of other buffers). It is accounted for in the 'inactive memory', because it's contents can be at any time thrown away (and the memory made 'free'), and about the only cost would be to re-read the same data from disk/network again.

Again, it is my understanding that the so called "disk cache" (not to be confused with the I/O buffers managed by the kernel) is stored in the speculative memory (that's what the documentation in the source code says). Which is well separated from the Inactive memory. When after an hard fault a page needs to be read from the disk (like when loading the process code) the kernel reads in advance a chunk of contiguous pages, loads in the active memory the addressed page and puts the rest of pages in the speculative memory.

There is also a pretty complicated aging mechanism to have the speculative pages released/replaced in content after some time for sure used at least by the pageout daemon. Which is the easier to understand. Unfortunately the pagein process seems to be embedded in the fault manager and that is very complicated (to me) to extract it from there.

The best thing would be to make some small programs which only purpose it so allocate/read/write large chunks of memory and monitor the memory while they are executing. Unfortunately Xcode for me is too much a big tool just for that. I am pretty suprised it doesnt come with a simple "cc" command line compiler (unless of installing Macports, but I don't feel like that either). I'll look around, I may find something

Warwick Teale wrote:

Thus if that assumtpion is true, then WHY is this behaviour/artefact of this ALLOCATED UNUSED (correctlabel or not), is exaccebated in recent OSX LION 10.7.x maint releases?

Or to put it another way, why was this behaviour exhobiated in pre OSX 10.7 flavours of the OS?

I feel there is a defect in there some where in the recent OS.

I don't know. If my explanation is correct it may well depend on the last releases of the applications as well. If you have the chance you should try running the same applications in Snow Leopard. Otherwise google is your friend. You may search if those applications are reported as memory hungry (or leaking). The OSes have little defense against this. Any application is allowed to require all (and even more) available RAM in the system. Unless of putting in place some kind of quota mechanisms but they would just be fictitious workarounds.

dkalchev wrote:

Your other observation, and right so (matches mine) is that typically third party software allocates lots of memory they never use, or in general have suboptimal memory footprint. This is, directly speaking sloppy programming!

FWIW, the Caching and Purgeable Memory developer doc talks about the benefits & pitfalls of using large memory caches. It isn't sloppy programming to allocate lots of memory if it improves performance (which is often true), & if it is done using the methods mentioned in the doc, it doesn't have to have much of an effect on other processes.

Michelasso wrote:

Again, it is my understanding that the so called "disk cache" (not to be confused with the I/O buffers managed by the kernel) is stored in the speculative memory (that's what the documentation in the source code says). Which is well separated from the Inactive memory.

Haven't had time to read the source code -- my opinion is based on how things are done in other BSD UNIX systems, such as FreeBSD (although in FreeBSD things have become much more complex).

Quick Google on 'os x buffer cache' came to this article, which has some good explanation. Perhaps way better than mine, as English is clearly not my native language.

http://reviews.cnet.com/8301-13727_7-57372267-263/purge-the-os-x-disk-cache-to-a nalyze-memory-usage/

This acticle also explains in plain language, what people have been mystyfying about: the purge command.

So yes, apparently the buffer cache (aka Unified Buffer Cache) is displayed as "Inactive" memory.

It is normal for this memory to grow very fast when you read in large files (multimedia, backups) and not "free" immediately when you stop the application. For it is not memory allocated by the application itself.

Another issue is with memory mapped files, that are adding to the pagein counters, without being actually RAM related. (Mention this, because I once had huge number of pageins in Activity Monitor, with no page outs and no swap allocated)

Michelasso wrote:

Otherwise google is your friend. You may search if those applications are reported as memory hungry (or leaking).

If you do that, you will find complaints about just about every large app being excessively memory hungry or leaking, including ones running on any version of Tiger, Leopard, & Snow Leopard you can think of. Some users seem to have the idea that developers can somehow maximize performance & feature support without increasing memory demands, but that isn't very realistic.

R C-R wrote:

FWIW, the Caching and Purgeable Memory developer doc talks about the benefits & pitfalls of using large memory caches. It isn't sloppy programming to allocate lots of memory if it improves performance (which is often true), & if it is done using the methods mentioned in the doc, it doesn't have to have much of an effect on other processes.

Absolutely! But not all applications make use of the frameworks and APIs provided by Apple. This does not apply to OS X only of course -- many multi-platform toolkits attemt to do their own 'smart thing', often the same way on different operating systems, which is obviously wrong concept. But.. a lot of people subscribe to the idea, that 'memory is cheap' and programmers still write programs that asume: unfinitely fast CPU, infinite amount of RAM and disk. Anyway, this is my observation, for many different kinds of software.

dkalchev wrote:

Quick Google on 'os x buffer cache' came to this article, which has some good explanation. Perhaps way better than mine, as English is clearly not my native language.

http://reviews.cnet.com/8301-13727_7-57372267-263/purge-the-os-x-disk-cache-to-a nalyze-memory-usage/

This acticle also explains in plain language, what people have been mystyfying about: the purge command.

So yes, apparently the buffer cache (aka Unified Buffer Cache) is displayed as "Inactive" memory.

That article, although mostly correct, makes the terrible mistake of using Activity Monitor as the monitoring tool, which is incomplete. The only reliable one is vm_stat. Here an example of what it happened using "purge" in my system (I just done it now):

free active spec inactive wire faults copy 0fill reactive pageins pageout

48454 470013 50451 102894 114303 4516 161 3431 0 115 0

226182 391081 5407 48328 114466 2914 833 901 0 284 0

This is an aproximation of the delta values:

free: +178,000

active: -79,000

speculative: -45.000

inactive: -54,000

wired: 0

The active memory had the largest reduction. The speculative and inactive one about the same, with the speculative the highest in percentage. Thus purge does more than what it is described in the purge man page. The conclusion is right, though. To use purge has little benefit. Actually it makes the system slower. Those pages get soon reused at the cost of having eleminated all disk caches (and whatever else).

Anyway I am surprised that R-C-R didn't correct you already: XNU, the OS X kernel, is not a FreeBSD kernel. It's an hybrid one. It consists on the Mach microkernel (to which the virtual memory belongs) with components from FreeBSD and the I/O kit for writing drivers. Thus I am afraid that what you know about FreeBSD memory management must be revised when talking about OS X.

R C-R wrote:

Michelasso wrote:

Otherwise google is your friend. You may search if those applications are reported as memory hungry (or leaking).

If you do that, you will find complaints about just about every large app being excessively memory hungry or leaking, including ones running on any version of Tiger, Leopard, & Snow Leopard you can think of. Some users seem to have the idea that developers can somehow maximize performance & feature support without increasing memory demands, but that isn't very realistic.

Like everything Google must be used with a grain of salt. One has to select the relevant information.It did happen to me to find technotes or patch changelogs that talked about memory leaks in a couple of applications. As it is common knowledge, for example, that AdBlock and Safari for OS X are memory hungry.

Michelasso wrote:

Anyway I am surprised that R-C-R didn't correct you already: XNU, the OS X kernel, is not a FreeBSD kernel. It's an hybrid one. It consists on the Mach microkernel (to which the virtual memory belongs) with components from FreeBSD and the I/O kit for writing drivers. Thus I am afraid that what you know about FreeBSD memory management must be revised when talking about OS X.

I already know this. Which is why I try to dwell as little as possible into specifics and direct comparisons.

In any case, the differences in the VM principles are not that significant. Terminology does differ though.

You did your purge test, got some results. What is your explanation of those observations?

Do you still believe that buffer memory is speculative memory? What was freed form the 'active' memory?