Memory used by unknown processes

Question

erlichg Author

Level 1

1 points

Memory used by unknown processes

Hey all,

I have a serious problem on my 2008 model macbook running snow leopard 10.6.8 with 4GB of memory

Used memory continuosly grows even though no work is being done (i.e. no application opened).

It gets to a point where free memory goes almost to zero and only then system apparently frees some of it.

Using free memory app I can free memory manually but still after freeing, free memory continue to decline.

What really bothers me is that the sum of all memory used by processes (from both activity monitor and ps command) does not sum up to total used memory.

Can some "invisible" process be using my memory? or ps/activity monitor outputs are not correct?

MacBook, Mac OS X (10.6.8)

Posted on Jan 26, 2012 1:50 AM

Reply

Answer 1

X423424X

Level 6

14,271 points

Jan 26, 2012 2:09 AM in response to erlichg

Make sure you show all processes with activity monitor and/or ps ax. You might also want to use top with an appropriate -o option (rsize?).

Reply

Answer 2

erlichg Author

Level 1

1 points

Jan 26, 2012 3:46 AM in response to X423424X

Regarding activity monitor - it is.

As for ps, I'm using ps -axm -o "rss,comm"

Ans as for top, I thought it shows the same proceses as in ps, or am I wrong?

Reply

Answer 3

R C-R

Level 6

17,794 points

Jan 26, 2012 4:45 AM in response to erlichg

erlichg wrote:

... or ps/activity monitor outputs are not correct?

From the man page for ps:

"Since ps cannot run faster than the system and is run as any other scheduled process, the information it displays can never be exact."

Also note the "PERFORMANCE vs. ACCURACY" info from the man page for top.

IOW, to oversimplify somewhat, memory use can change faster than the monitoring tools can track, & the more accurate they become the more they use resources that can't be used by other processes.

Reply

Answer 4

erlichg Author

Level 1

1 points

Jan 29, 2012 12:01 AM in response to R C-R

I'm not looking for exact to the last byte, but the difference between sum of all memory shown by ps and the actual used memory is around 1GB!!

Reply

Answer 5

fane_j

Level 4

3,704 points

Jan 29, 2012 2:02 AM in response to erlichg

erlichg wrote:

the difference between sum of all memory shown by ps and the actual used memory is around 1GB

In my sheer ignorance, I believe that the ~1 GB is the wired memory.

As I understand it, the sum of the RSSs displayed by ps is equivalent to Activity Monitor's Active memory. The Wired memory, ie, "kernel code and data structures that must never be paged out to disk", would not show up with ps, because user-level processes cannot allocate wired memory, although they affect it by requiring kernel resources. Of course, I may be very wrong. But, if I'm right, 1GB is a little on the high side -- I'd expect something closer to ~700MB for a 4GB system.

Anyway, if I'm right, then don't waste your time seeking invisible processes; rather, track periodically the active memory with ps to find out which process is the leaky one.

Reply

Answer 6

erlichg Author

Level 1

1 points

Jan 29, 2012 3:43 AM in response to fane_j

Not exactly sure what you mean.

Here's output from top (the header):

Processes: 74 total, 3 running, 1 stuck, 70 sleeping, 535 threads 13:40:42

Load Avg: 0.36, 0.22, 0.35 CPU usage: 8.33% user, 41.66% sys, 50.0% idle SharedLibs: 5888K resident, 8800K data, 0B linkedit.

MemRegions: 13884 total, 1746M resident, 28M private, 380M shared.

PhysMem: 590M wired, 2219M active, 1025M inactive, 3834M used, 261M free.

VM: 178G vsize, 1043M framework vsize, 738071(0) pageins, 235(0) pageouts. Networks: packets: 660846/677M in, 332007/37M out.

Disks: 327320/5791M read, 154559/5840M written.

And here's the sum of RSS from ps:

# ps -axm -o "rss,comm"|grep -v COMM|awk 'BEGIN{a=0}{a+=$1}END{print a}'

2537976

As you can see, top reports 3834MB as used, while sum of RSS from ps shows ~2.5GB

Reply

Answer 7

X423424X

Level 6

14,271 points

Jan 29, 2012 4:19 AM in response to erlichg

I am not sure how to interpret all the numbers but I am pretty sure your interpretation of them is way off base. I just did the exact same operations you did (your ps command line and a top). My top's "used" value was 5452M. My ps command yielded 2067976, i.e., ~2068MB (2.068GB). So according to your math there is 3368MB (5452MB - 2068MB = 3368MB) of unaccounted for memory and I don't believe that for a moment.

Reply

Answer 8

erlichg Author

Level 1

1 points

Jan 29, 2012 5:06 AM in response to X423424X

So one of the 2 commands is not what I think it is.

but which one?

The busy + free numbers from top is the exact sum of my memory, so they seem to be right.

This means that ps output is wrong, or atleast the RSS column.

Assuming top and activity monitor are right, I have 3834M of used memory and now I need to find out what are the processes that use it...

Reply

Answer 9

fane_j

Level 4

3,704 points

Jan 29, 2012 5:55 AM in response to erlichg

erlichg wrote:

Not exactly sure what you mean.

I'm sorry, I misunderstood what you wrote. My understanding (which may be wildly wrong) is that

sum of ps RSSs ≈ top active + inactive

So now I see that your wired memory is ~0.6GB, which is reasonably close to what I expected. Active + inactive is 3.2GB, while ps reports 2.5GB (assuming your awk syntax is correct, which I have no reason to doubt).

So there is a discrepancy of ~0.7GB. I've no idea how to account for it. Keeping in mind that I could very well be wrong, my suggestions are, (a) use something else to sum RSSs reported by ps, just to double check the result printed by awk, and (b) could it have anything to do with the stuck process? Stuck does not necessarily mean hung, but it might be worth looking into it.

Reply

Answer 10

erlichg Author

Level 1

1 points

Jan 29, 2012 6:26 AM in response to fane_j

hmm, I've summed the RSS in all ways I can think and still come up with the same answer.

Don't know what the stuck process was, but now it's gone. Things haven't changed :-(

Reply

Answer 11

fane_j

Level 4

3,704 points

Jan 29, 2012 4:41 PM in response to erlichg

erlichg wrote:

Don't know what the stuck process was, but now it's gone.

Then it was normal. Sorry, I'm fresh out of ideas. I guess the most likely explanation is that I misunderstand the relation between memory reported by ps and, respectively, by top. Perhaps someone else can explain it…

Reply