How is it possible to compress so much memory in RAM?

How is it possible to compress so much data in RAM but not be able to compress normal files this much?User uploaded file

iMac, macOS High Sierra (10.13.4)

Posted on Aug 7, 2018 9:57 AM

Reply
14 replies

Aug 7, 2018 4:47 PM in response to Student_Learner

Memory compression is parts of an app, not the entire app. App data in memory often times has lots of zeros. For example, 64-bit pointers often have the first few bytes always zero or some other constant value that is ideal for compression. Partially filled buffers may have zeros in the unused parts. Buffers containing text is highly compressible.


Also, you are seeing megabytes, not gigabytes of compression.


As I said earlier, the really big files tend to be music, pictures, and movies, all of which are already highly compressed, generally using compression models tailored to the kind of data and repeat patterns often found in the media.

Aug 9, 2018 5:47 AM in response to Student_Learner

There seems to be some confusion about what the compressed memory is - it is application data that hasn't been used recently, and is compressed rather than swapped out by the VM. Without knowing exactly what the application data is (caches, data structures, etc), it will be hard to determine what kind of compression percentages apply. I think in your screenshot it is just coincidental that some of the memory (application memory) and compressed memory (application data) columns are similar - looking at other processes, there are many with no compressed memory, and some with compressed memory much larger than the application memory, so I think it just depends on the application and what it stores in memory.

Aug 8, 2018 8:10 PM in response to Student_Learner

From the webpage pointer provided above it says compression cuts the size in half. And the compression algorithm is WKdm.


...In Mavericks, the OS has one more option before it has to resort to swapping: compressed memory. Mavericks will find the least-recently-used data in memory and compress it, usually to about half its original size. Et voilà, more free memory.

Like the HFS+ compression feature introduced in Snow Leopard, compressed memory trades relatively abundant CPU cycles for decreased disk I/O. The compression and decompression process is extremely fast, using the WKdm compression algorithm, and it will run in parallel, taking advantage of multiple CPU cores...

Aug 8, 2018 6:20 AM in response to Student_Learner

My assumption is that lsd is using 269.9MB of memory and that of that memory 268.5MB is compressed. That is to say 258.5MB is compressed code and data, and 1.4MB of that memory is uncompressed. That tells you nothing about how much memory lsd would be using if it was all uncompressed. You also do not know if all of the lsd code is loaded into memory, nor how much memory lsd has allocated for data and stack space.


To put it another way, I have a "Jack-in-the-Box" with a compressed spring. I know how big the box is (resident memory), but until I open the box (uncompress it), I do not know how much space the expanded Jack-in-the-Box spring will occupy.


You and I are both drawing conclusions based on partial information.

Aug 8, 2018 3:48 AM in response to rccharles

I think the poster wants an explanation of the difference in the numbers between the memory and compressed mem. What does a zero mean? I assume it means no compression was done. What does a memory number of 269.9 vs. 268.5 mean? Is it that 269.9mg got compressed into 268.5?

Since my system has all zeros for the Compressed Memory column, I would assume that 0 means nothing was compressed.


In the examples given by Student_Learner and rccharles, I'm going to guess that the Memory column is the total resident memory the process is consuming (both uncompressed and compressed), and the Compressed Memory column is the subset of that resident memory containing compressed code and data. This is a guess, but in the examples shown, the Compressed memory is always less than the resident memory, even if it is by just a tiny bit. If resident and compressed were not related this way, then there would be examples of compressed being larger than resident memory. Again, just a guess based on what I'm seeing.


I also assume that if that program scheduled for execution, any pages the program wants to access which are compressed will first have to be uncompressed before the program can use that page of memory.

Aug 8, 2018 3:54 AM in response to BobHarris

Actually, I thought my question was stated simply enough. In the above example the "lsd" algorithm is using 269.9 total memory of which, 268.5 is compressed. My question is, how can you even "compress" all that data (268.5 MB) when every other normal compression algorithm cannot achieve that. Say if compressed a word document, I would never attain near 99% compression of the file or any other for that matter.

Aug 8, 2018 6:52 PM in response to BobHarris

I am drawing my conclusion based off of compression algorithms. If your going to call it compressed memory, its pretty much understood that it involves a compression algorithm. And there isn't any compression algorithm that can achieve that much data compression. That's over 99% compression. So, if it is "truly" compressed data, how is it achieved. The more likely explanation, is that is not true compression. Every compression program, algorithm, app, or whatever you wish to call it cannot achieve that amount of compression. The only way to even near that compression would be to compute in base 3, ternary, or trinary...again...whatever you would like to call it.

Aug 8, 2018 6:57 PM in response to leroydouglas

I don't just visit these forums if I have a problem that needs to be solved. I also visit these forums to learn and as such I think my question is a valid one and a correct answer would make a lot of difference in expanding my knowledge. If that irritates you to where you have to ask me "What difference does it make?", then don't bother posting any responses.

Aug 7, 2018 1:06 PM in response to Student_Learner

I do not understand your question.


The screenshot you show, says your memory is overcommitted 119MB which has been moved to a swap file. Since your Memory Pressure is not yellow, you are most likely not compressing memory in RAM at the moment.


As to compressing files, the worst offenders for filling up a disk are often music, picture or video files which are already stored in a highly compressed form, so it is not really possible to compress them further.


Can you be more specific about what you are asking?

This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

How is it possible to compress so much memory in RAM?

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.