1 25 26 27 28 29 Previous Next 957 Replies Latest reply: Dec 1, 2013 1:28 PM by Jonathan Payne1 Go to original post
  • 390. Re: Lion - Memory Usage Problems
    albert421 Level 1 Level 1 (0 points)

    I just bought a new Mac Mini with 2GB RAM, and it is slow. So I upgrade it the 8 GB, but if I would just run safari with only one tap, and the new Parallels Desktop 7 with Windows 7, and I give Windows 7 1.5GB RAM. After it run for half of an hour. There is only 100MB FREE RAM, Wired 3...GB, only 1.5GB are active and more than 3GB are inactive. Why are there so much of the RAM are INACTIVE?

  • 391. Re: Lion - Memory Usage Problems
    R C-R Level 6 Level 6 (14,175 points)

    Gabriel Couto wrote:

    I just wanted to report that I think my memory use in Lion is not normal. I have around 5 programs running(dashcode, textedit, excel, safari and sparrow)... almost none of them is eating memory, except Safari

    You also have a lot of add-ons running, like Dropbox, GrowlHelper, Alfred, etc., & some things I don't recognize, like talagent or VTDeoderXPCservice. If you are experiencing any performance problems, you should make sure that these items are up-to-date, compatible with Lion, & so on.

     

    But you don't seem to have any performance issues related to memory use, as evidenced by the low page out statistics in the Activity Monitor screen shot. If that is true, & you are just concerned about the amount of memory Safari is using, consider the following:

     

    1. It is the history of the web pages you have opened since launch that determine what Safari & Safari Web Content have loaded into memory, not just the currently open web pages. Since what is in memory at some point might be reused (for instance, if you open a web page again), the OS won't automatically free up the memory used for one web page when you close it unless that memory is needed for something else. Instead, some or all of it will just be marked as inactive, but still contribute to the Real Memory shown for the process in Activity Monitor.

     

    2. Displaying even simple looking web pages may use a lot of memory. A web page isn't a document. It is basically a set of instructions (a markup language written in SGML, XHTML, etc.) & the data those instructions pertain to that the browser assembles into the rendered web page you see on your screen. Because of this, substantial memory may be needed not just to hold the displayed page but also the code & working space needed to render it. For instance, if the page source includes compressed image files, the browser needs memory for the file info, for the uncompressed images, & for any associated buffers or other working space its rendering/layout engine (like WebKit or Gecko) requires.

     

    3. Because of the above, you can't tell just by looking at the rendered page how much memory it used, or for what. To do that, you need to look at the page source & have some familiarity with markup languages, scripting, & anything else involved in rendering the page. And comparing the difference among browsers won't tell you much, even if they use the same layout engine & support the same features. Even more than with almost other type of app, browser designers must make tradeoffs among performance, memory use, & layout & other features supported. Each of these design decisions strongly influences what should stay in active RAM, what should be moved to the inactive list, & what should be freed up & when.

     

    An obvious example: Safari's "Top Sites" feature. Those pre-rendered pages need memory space somewhere, & if RAM is available it will be used.

     

    4. Because of all this, the more RAM you have installed, the more of it the system will use. There is nothing to be gained by freeing up RAM unless & until it is needed for something it doesn't already contain. It would be ridiculous to discard something that might be reused & have to fetch it or build it again if it was, just so curious users would see a larger free memory segment in Activity Monitor's pie chart or a smaller Real Memory number for some app. RAM is meant to used. If it wasn't, it would be stupid to sell Macs with so much of it installed or to install more.

     

    5. To understand why or when RAM should be freed up, you must consider all the processes running, not just one or two isolated application processes like Safari or iTunes. Since the OS isn't going to release RAM until it needs to, if other processes are not asking for so much RAM of their own that the total demand is for more RAM than there is available, the OS isn't going to free up any of it. (The exceptions are when an app quits or explicitly asks the OS to free some of the memory it has been using.) And since the OS is running its own set of processes on its own schedule independently of what the user does, this total can change without warning (& cause spikes too fast for Activity Monitor to track).

  • 392. Re: Lion - Memory Usage Problems
    Michelasso Level 1 Level 1 (65 points)

    R C-R wrote:

     

    1. It is the history of the web pages you have opened since launch that determine what Safari & Safari Web Content have loaded into memory, not just the currently open web pages. Since what is in memory at some point might be reused (for instance, if you open a web page again), the OS won't automatically free up the memory used for one web page when you close it unless that memory is needed for something else. Instead, some or all of it will just be marked as inactive, but still contribute to the Real Memory shown for the process in Activity Monitor.

    Let me understand, because this point of yours has never been clear to me. Are you saying that Safari is practically caching in RAM the Internet content of the old pages?

     

    If so, that would be absolutely wrong. What a page loads at one time could be completely different to what it will load in a second time just doing a reload. That's also why we reload the pages indeed. There is no certainty whatsoever about the data coming from the outside. Anyone can change anything at any moment.

     

    I hardly see how Safari or any web browser could reuse any data it downloaded previously. Let's suppose a page contains a sort of plugin to display a video. The web browser allocates a memory area for the code needed to run the video. Cool. Then we reload, Safari finds what it thinks it is the same video with the same video player but in the mean time the web master updated code because of a bug playing he new video and content. Safari runs the old code it kept in memory and the video play breaks. Or it plays the old video, which is clearly not what we want. That would not be acceptable. The content stored in the application memory would be just garbage. What's the point to keep it, then?

     

    Unless I misunderstood, sure. But the way Safari keeps growing in memory doesn't look natural. Or well programmed, whatever we wanna call it. Having a slow internet connection, like mine, to cache data in RAM is just a waste of resources with no benefits. While downloading a page the applications would have all the time to reload from disc what needed, if any. Still I don't understand how some people report Safari Web Content process using only 150MB with 15 or more tabs open after few hours working. I've seen the Actvity Monitor screenshots, I couldn't believe my eyes. Mine goes to 300MB just after launching it.

  • 393. Re: Lion - Memory Usage Problems
    R C-R Level 6 Level 6 (14,175 points)

    Michelasso wrote:

    Welcome to the nightmare of Virtual Memory Management in OS X!

    It is a nightmare only because it is very difficult to understand how it works it without a comprehensive study of the technical documentation intended for developers writing code for OS X, not for end users. Contrary to what some of those users believe, you cannot just look at a few statistics provided by Activity Monitor or similar high-level tools & determine if some process is using "too much" RAM.

     

    Good memory management is all about trying to keep information in RAM for as long as it remains useful. Some users seem to have gotten that completely backwards, thinking that it is all about maintaining as much unused (free) RAM as is possible. But it isn't about that at all. Free memory is nothing more than the amount of RAM that is not currently being used for anything. Its only value is it is unallocated to any process & thus immediately ready to be used for any processes that need more RAM than is currently available. But as long as there is enough available, it has no value. It is that simple, & I don't really see how anyone could get that part so wrong.

     

    What isn't simple is understanding what happens when there isn't enough RAM available for all the running processes. The memory manager can do several different things to make enough available. Both what it actually does & how long it takes depend on several interrelated factors, including how each individual process is using memory. If you just know how much they are using, you do not know enough to say if it is too much or not.

  • 394. Re: Lion - Memory Usage Problems
    Michelasso Level 1 Level 1 (65 points)

    R C-R wrote:

     

    Good memory management is all about trying to keep information in RAM for as long as it remains useful. Some users seem to have gotten that completely backwards, thinking that it is all about maintaining as much unused (free) RAM as is possible. But it isn't about that at all. Free memory is nothing more than the amount of RAM that is not currently being used for anything. Its only value is it is unallocated to any process & thus immediately ready to be used for any processes that need more RAM than is currently available. But as long as there is enough available, it has no value. It is that simple, & I don't really see how anyone could get that part so wrong.

    Actually the only part that everyone is getting right is exactly the one about free memory. If it isn't needed, just don't use it. You seem to say the opposite: use as much free mem as you can because "free memory is wasted memory". Inactive memory might be reused in another moment. The question is: how? To me in many cases you mentioned (like the web browser's old pages) that's impossible. The only possible use for it is to rewrite its content. So keeping the memory allocated is the waste, not an high free memory,

     

    That's where we disagree. Useless allocated memory is garbage and garbage as we know must be taken out of the house. You claim that it isn't useless because it may be reused. I say that I highly doubt that. The numbers Lion is currently using often are far too high for a casual use. 2.8GB of swapped data in a single shot is not "free memory is wasted memory" it is trashing my and other systems badly. Very likely a bad big bug.

     

    I checked the OS X virtual memory's source code in the Open Source Apple site. To say that it is complex is a huge understatement.It would take a month of studying the code to have a partial understanding. Looking at the comments and the code one wonders even more what those values (active, inactive, free, speculative, etc) really are. The module seems to do much more than what briefly described in the Apple documents.

     

    The complexity and the big size of the code also make the chances of bugs much higher than what I thought. With all the bugs infesting Lion I don't see why the VM module should be immune. You seem not to want to even consider this option, no matter how many weird behaviors have been reported in this thread.

     

    At this point I am left to think that your system doesn't trigger any of those bad behaviors. It may happen. Yesterday I went to the mall looking for an HDD. There was the Apple section so I tried the new MBA. How in Earth is that possible that Safari took only 85MB and Web Content 65MB when my sessions always start with at least 120MB+250MB with one tab? And I manually deleted caches, history files, everything a couple of times. No changes.

  • 395. Re: Lion - Memory Usage Problems
    R C-R Level 6 Level 6 (14,175 points)

    Michelasso wrote:

    Let me understand, because this point of yours has never been clear to me. Are you saying that Safari is practically caching in RAM the Internet content of the old pages?

    It isn't that simple. A web page is really just a series of elements & a set of instructions that tells the browser what to do with them. (Safari has several tools to show you this, including View Source, the Activity window, & the various 'show' items in the Develop menu.)

     

    So if say a browser encounters an instruction to run a Javascript, then the browser must load & initialize its code to do that, create any buffers or other work space that task needs, & so on. Some of the work is passed on to OS level API's, but it still has to set up the variables & such to pass to the API, gather any info or signals the API passes back to it, etc. Likewise, an instruction might say to fetch a series of image elements (pngs, gifs, jpegs, whatever) & display them, link some of them to button scripts, & so on.

     

    All of this requires memory. Some of it can be reused. For example, if some of the static image elements (like say the ones that are used to construct the Apple header at the top of this page) are used on multiple pages, then they don't need to be downloaded & reconstructed on each page view if they are still in memory from a previously viewed page. The same goes for initialized code elements that run the instructions. Only the elements that are unique to the web page, change on reload, or have not previously been encountered must be processed before the new page can be rendered.

     

    This is a very simplified explanation of a topic that can fill books so you will need to consult other sources to understand the details of how it actually works, but the essential point is that web pages are built in memory from many different parts & some of them can be reused, along with the code used to build them. This goes much faster if these elements are already in real memory (RAM) than if they have to be reloaded from the backing store, from system files, or downloaded again from the Internet, so the memory manager will try to keep them in RAM if there is room for that.

     

    Having a slow internet connection, like mine, to cache data in RAM is just a waste of resources with no benefits.

    On the contrary, if every element of a web page had to be downloaded every time it was encountered again, web pages would take much longer to load over a slow connection.

     

    Still I don't understand how some people report Safari Web Content process using only 150MB with 15 or more tabs open after few hours working. I've seen the Activity Monitor screenshots, I couldn't believe my eyes. Mine goes to 300MB just after launching it.

    As has been mentioned here many times, the amount of real memory an app uses depends on how much RAM is installed in the Mac, on what tasks the app has done previously, & on the demand for real memory from other running processes. Those other processes include both other apps launched by users & various root & other non-human user processes that run on their own schedule. Most processes are "faceless" (no UI) & may support per-user or system-wide tasks that other processes require.

     

    Complicating things that much more, processes may share memory, which can be inherited from a parent process, read from or written to by two or more processes, & used for things like fast interapplication communication or sharing icons or other resources.

     

    If you don't understand & consider all these interrelated factors -- or have the details necessary to do so from the various reports -- there is no way you will be able to tell if Safari (or any other app or system process) is using memory correctly.

     

    The numbers you get from Activity Monitor & similar tools only tell you a small part of what you need to do this. You can't just look at say the real memory used by a process & know if it is too large or not. To determine that, you need to know the details of what the process does & how it does it, how it asks the OS to allocate & free whatever memory it uses, what other processes might also be sharing a part of its memory or asking for memory of their own, & all the rest of it. Much of this requires the use of tools that actually show how memory is used within a specific process; some of it requires access to the source code of the app or at least its symbol table. Otherwise, you basially are trying to figure out how something works without any idea of what its internal structure is or what its parts do.

     

    If this were about some other complex device, say a car or TV, I think it would be obvious to most people why this is so. I don't understand why some people think it is any less applicable to a computer, arguably the most complex device most of us ever use.

  • 396. Re: Lion - Memory Usage Problems
    Michelasso Level 1 Level 1 (65 points)

    All right, thanks for the explanations. Still few points are obscure to me. I have the tools to understand how the VM works, but I know nothing about Web programming. Let's see:

     

    • You said that when we load a web page some data can be static, so much that it can be reused? Let's take the Apple header. Apple can change it just now. Or there is a Web programming convention that a frame can be marked as static and therefore reused?
    • Still I'd see how that could be cached on disk using the URL as "key". Hardly in RAM. The VM module has no references to the URLs to be able to map and thus reuse them. Unless sure, the web browser implements and manages its own URL caches in RAM.
    • The question tho still remain: even assuming that's the case, what does it happen to the web RAM cache when a web page gets changed? That memory should be discarded.
    • Together with the above one wonders how efficient that would be, Many web pages are accessed only once. The web browser at that point should implement a garbage collector, and as we well know that for sure isn't the case in Safari. So we have to assume that is what makes the inactive memory growing in large sizes.
    • The Activity Monitor screen shot that I reported with low Safari memory consumption showed 4 cores. Not the memory pie chart so I can only guess the physical memory. Knowing the Mac models, 4 cores should mean at least 4GB which is above my current RAM (3GB).
    • I tried a MacBook Air 11", yesterday and Safari (loading only 4 tabs with different Apple site pages) used a total amount of 150MB. 85MB + 65MB. It had 2GB RAM only, but still.. I have never seen anything like that before. I wanna try to do the same with a fresh guest account.

     

    Regarding shared memory, just to be sure we speak the same language:

    • The idea behind it is to save memory. For example the shared libraries: sharing the same code the RAM for them needs to be allocated only once.
    • When forking and then writing in a shared memory page the copy_on_write() system call usually gets called. The memory at that point is duplicated and assigned to the process.

     

    I agree it is not trivial to retrieve the right image of what it is really on memory. The tools we have available either give a general idea (vm_stat) or they are far too detailed (vmmap). Still to complitely analyze that would be too deep in this discussios and probably out of topic. At the end Activity Monitor gives a fair enough description. Some numbers may look funny but that's ok.

     

    What I really care is to point out that having only 3GB I don't have many problems if Lion once in a while pages out some memory. I am not happy with Safari high memory usage, but I could deal with few megaytes swapped out now and then. If that RAM we assumed Safari cached from the web sites doesn't get reused (simply because we don't access those web pages anymore), it would become inactive memory and then swapped out when some more free memory is needed. Fine. There would be just little overhead writing the pages out. Being most of that memory nearly never reused the swap files would just grow in size and stay there quite until we quit Safari or reboot. Nothing wrong with that. I could even deal with 50-100GB of swap files, that wouldn't affect anything.

     

    But again, that is not what happens. In these weeks I just saw the dynamic_pager getting crazy. Paging out even gigabytes just doing nothing more than writing in this forum. The inactive memory keeping growing nearly using the same web sites. Page outs after one hour of usage. Large/huge pages outs after a couple of hours.

     

    Once after a 2.7GB swap out I logged out and in again. The memory monitors showed the usual after login values, the swap used was 0, but the system was unusable. Slow, stuttering. I had to reboot. That is a bug to me since Lion is always very efficient. Until it pages out large chincks of RAM, that's it.

  • 397. Re: Lion - Memory Usage Problems
    R C-R Level 6 Level 6 (14,175 points)
    Michelasso wrote:
    That's where we disagree. Useless allocated memory is garbage and garbage as we know must be taken out of the house. You claim that it isn't useless because it may be reused.

    Neither can you claim that everything becomes garbage just because it has been used once.

     

    If say a browser has to load & initialize its code in real memory to handle some specific task, it would be a very poorly designed system if it had to do that over & over again every time it had to perform that same function again. And since many tasks use the same code fragments, buffers, data tables, & so on, it would be supremely stupid to purge all those pieces from real memory after every use just to make room for what will most likely be a lot of the same pieces needed to handle the next task.

     

    Likewise, if a web page includes some elements that other web pages also include (like the graphic elements of the headers & footers of this page) why would you want to have to download them over & over every time you visited a web page that used any of them?

     

    If nothing else, consider how miserable it would be if every time you used a browser's 'back' or 'forward' buttons it had to download every single element of those pages all over again & re-render the whole page from scratch.

     

    I think what you might be missing is how granular so many things are. A lot of memory objects are no larger than a few 4 KB pages, some don't even fill one. That makes it possible to load only what is needed to do some task from disk, to keep the reusable items in RAM without having to do the same for everything else, & to find room for new items as they are needed.

  • 398. Re: Lion - Memory Usage Problems
    William Kucharski Level 6 Level 6 (14,695 points)

    Generally, the way the cacheing would work is that the browser would control it, with the content being stored locally in cache files.

     

    The contents of the most recently used cached files would also be disk blocks cached in RAM by the file system and the VM system.

     

    Thus content that can be reused may still be in RAM.

     

    If Apple changes content, there is HTTP information that will be exchanged so the browser knows this is new content and to discard the cached information.

  • 399. Re: Lion - Memory Usage Problems
    Michelasso Level 1 Level 1 (65 points)

    William Kucharski wrote:

     

    Generally, the way the cacheing would work is that the browser would control it, with the content being stored locally in cache files.

     

    The contents of the most recently used cached files would also be disk blocks cached in RAM by the file system and the VM system.

     

    Thus content that can be reused may still be in RAM.

     

    If Apple changes content, there is HTTP information that will be exchanged so the browser knows this is new content and to discard the cached information.

    Yup, that is correct. I've found a good explanation about how Web Caching works. HTTP basically informs what it is cacheable and what a web page content has been changed, so much that a web browser will reload the modified/expired content when needed. Well, it makes sense.

     

    Still that is a disk cache that can be stored in the I/O buffer. As long as Safari reloads the same pages, their disk cached content will be kept in the buffers.What is it stored in the VM that Safari preferes not to discard?

     

    I know, I am annoying. But more than 1GB or RAM (and growing), just to surf the web seems far too much to me. Also considering that all web pages content in RAM are modified pages that will be swapped out when the free memory gets low. To me it should keep in memory only the active web pages. I'd rather wait a little longer loading the pages than to fill all memory.

     

    Anyway, interesting enough I just found this link about Web browsers memory usage. I just gave it a fast look. Is it me or in Windows they are using even less than 1/3 of the memory used in OS X? I'll read it better now.

  • 400. Re: Lion - Memory Usage Problems
    William Kucharski Level 6 Level 6 (14,695 points)

    Most operating systems take a lazy approach to freeing up memory, so once again it doesn't matter how much memory anything is taking up until it's needed elsewhere.

     

    So for example, if there are cached disk blocks that are no longer needed, that memory isn't actually freed up until low water marks for free memory are reached, or you would need a constantly running thread that did nothiing but free up unused memory.

  • 401. Re: Lion - Memory Usage Problems
    R C-R Level 6 Level 6 (14,175 points)

    Michelasso wrote:

    I know, I am annoying. But more than 1GB or RAM (and growing), just to surf the web seems far too much to me. Also considering that all web pages content in RAM are modified pages that will be swapped out when the free memory gets low. To me it should keep in memory only the active web pages. I'd rather wait a little longer loading the pages than to fill all memory.

    1. What would be an acceptable amount of RAM to you? What would you base this on? For example, do you know anything about the resources needed for Javascript or HTML 5 support? What about unique features like Safari's Reader? If not, how can do do anything besides make a wild guess?

     

    2. I have no idea what you mean by "all web pages content in RAM are modified pages." Why do you think they must all be modified? By what? Please explain.

     

    3. What on earth is wrong with keeping inactive content in RAM to improve performance? Do you dislike the whole concept of active & inactive memory, or is this just something you would prefer Safari (& possibly other web browsers) didn't do?

  • 402. Re: Lion - Memory Usage Problems
    Sjazbec Level 4 Level 4 (1,665 points)

    Lion uses ram differently then prior OSX. Without explaining the technical background, i can assure you that this is in no way hindering your or your apps from benefitting of all the Ram in your machine.

     

    If your apps ran succesfully on Snow with 2GB of Ram ( Lions minimum and double the minimum of Snow , because Lion runs by default in 64bit mode while Snow ran by default in 32bit mode) , then they will still in Lion with the same amount. The more, the better is a natural thing.

     

    You complain that "all the RAM" is occupied  with just the handful of Apps open you mentioned, but have you de facto tried opening more apps and see if there are problems related to "low on memory" sort of errors ?

     

    Lion can "park" applications that have NOT  the focus, are not in foreground and are maximized ( fullscreen mode on their own virtual desktop ). Best this is understandable with iOS ( iphone/ipad/ipod ) apps - you can open all of them, but the memory is only fully available to the one you are currently working on.

     

    "Unused Memory is bad memory" is a golden Rule in Linux/Unix philosophy. Only Microsofts OS swaps all the time, even if 16GB are available physical. OSX does the exact thing the other way round. And Lion now adds the "Parking" feature. As soon as an app asks for Ram, Lion will take it from apps that are currently opened but not "active" in the foreground.

  • 403. Re: Lion - Memory Usage Problems
    Michelasso Level 1 Level 1 (65 points)

    R C-R wrote:

     

    1. What would be an acceptable amount of RAM to you? What would you base this on? For example, do you know anything about the resources needed for Javascript or HTML 5 support? What about unique features like Safari's Reader? If not, how can do do anything besides make a wild guess?

     

    2. I have no idea what you mean by "all web pages content in RAM are modified pages." Why do you think they must all be modified? By what? Please explain.

     

    3. What on earth is wrong with keeping inactive content in RAM to improve performance? Do you dislike the whole concept of active & inactive memory, or is this just something you would prefer Safari (& possibly other web browsers) didn't do?

    1. The amount Windows 7 is using? Did you check the link I posted about how the different browsers on Win7 and Lion use very different amounts of memory? Win7 uses as low as 1/3 of RAM used by OS X. Then I did the same tests. Safari on my system was using up to twice the memory showed for Safari 5.1 on Lion. Duplicating exactly the same web page. So what on Earth is cached? Does it take at least 40-50MB just to render the same page multiple times?

     

    2. I mean that the content of the VM pages has been marked as modified. In the paging and swapping algorithm that means they will be paged out when "stolen" to raise the free memory. From what I remember when I studied this stuff 25 years ago, they differ from the pages containing data mapped from the disk (like the code) which can be easily discarded if not used because in the disk already.

     

    3 It is wrong because it is degrading performances. One golden rule can be that "unused memory is wasted memory" but before that there is the rule "avoid disk access [for memory] as much as possible". Especially the writes that  take longer.

     

    But let's forget about web browsers for a second. Let's say that we have a small program for doing slide shows. The programmers can chose to either:

     

    a. Keep all pictures in memory because they may be shown again

    b. Once a picture has been shown deallocate its memory and allocate some for the new picture

     

    They are both legit. No memory leaks. Still a. can grow indefinetely. It can arbitrarely take all system memory, because it thinks to be cooler than the other processes. Obviously generating swap outs for each new picture once saturated the RAM. That would be bad programming the least. At least that's what they taught me at the Flinstones age of Computer Science.

  • 404. Re: Lion - Memory Usage Problems
    R C-R Level 6 Level 6 (14,175 points)

    Earlier I wrote:

    The numbers you get from Activity Monitor & similar tools only tell you a small part of what you need to do this. You can't just look at say the real memory used by a process & know if it is too large or not.

    A bit more on the difficulties of trying to use Activity Monitor as a diagnostic tool for memory use problems can be found here, specifically why it is not a good tool for finding memory leaks. It may also help clear up some of the confusion about real & private memory.

     

    It boils down to this: there are no shortcuts. If you don't understand the complexities of how memory management in OS X works -- & especially if you don't even understand what the per process numbers shown in Activity Monitor really represent or how they are computed -- then there is no way you can tell from these numbers alone if some process is using too much memory.

     

    Common sense alone should tell you that if that were possible then that's what programmers would do & there would be no need for the debugging tools Apple provides.

1 25 26 27 28 29 Previous Next