Where has all my RAM gone? (Super Slow!)

PART I: TEST TO DETERMINE IF MDS IS REALLY THE CULPRIT

Let's disable mds to see how your Mac runs without it. The goal here is not to leave you with no mds (because that would break the ability to search for lots of stuff on your Mac) -- you'll just disable it temporarily to see if your memory is freed and your performance issues go away. In other words, we're just confirming that the only problem you have is mds.

Open the "Terminal" application. You'll find it in the Utilities folder. e.g. in Finder, click "Applications" -> "Utilities" -> "Terminal"

From within the Terminal window, type the following command (this will shut down 'mds'):

sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

You will be prompted for a password, this is because you used the "sudo" command which allows the following command (in this case "launchctl") to be run with elevated privileges. If this was the first time you ever used the "sudo" command you get an extra lecture about using it responsibly (mis-using the "sudo" capability can really mess up your computer. No playing around with sudo!)

The password it wants is your login password (since you are an 'admin' user).

If you check the list of running processes in "Activity Monitor" you should see that 'mds' is no longer running.

Test your computer's performance and amount of available RAM. Did this fix the performance issues?

If so, you really don't want to leave it this way. We haven't fixed anything... we've just isolated the problem to 'mds' and verified that if 'mds' isn't running then the symptoms go away.

Two things to note:

1) 'mds' will not start the next time your Mac is rebooted (the launchctl command told the Mac's launch services that you don't want this process started anymore.)

2) You can tell 'mds' to restart by repeating the above command, but replace the word "unload" with "load". This starts mds and tells launch services that you do want this service started automatically.

sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist



PART II: REBUILD THE METADATA INDEX

Before continuing, re-enable 'mds' using the command above.

The normal way to repair an 'mds' service gone haywire is to rebuild it's indexes. Apple's prescribed way is here:

http://support.apple.com/kb/HT2409

But that's basically what I already had you do earlier in this thread. If you're not confident that your dragging & dropping of the Macintosh HD had the correct effect, then we could have you use the command line method. There's more typing, but there's also more control and ability to make sure it's done correctly.

To use the Terminal window (if you rebooted, you'll need to open another Terminal window) to rebuild your indexes, eject / remove any external volumes so that the only disk available on your Mac is the boot disk, then type these commands...

First, switch off Spotlight indexing by typing this:
sudo mdutil -a -i off

This switches the indexing off ("-i off") for the metadata stores on "all" (-a) mounted volumes. You should have seen a message that looks similar to this:

/:
Indexing disabled.

Second, erase all the metadata stores by typing this:
sudo mdutil -E -a

This "erases" (-E) the metadata stores (the searchable indexes created by Spotlight) on "all" (-a) mounted volumes.

Checkpoint: At this point, (1) the 'mds' process should be running, (2) the indexing should be off, and (3) the metadata stores have been erased.

Before re-enabling the index (which would cause it to immediately start rebuilding a new index), check the 'mds' process in Acitivity Monitor. Is it still consuming massive amounts of CPU or has it settled down?

You might also stop & restart 'mds' (use the "sudo launchctl unload -w ... " and "sudo launchctl load -w ... " commands listed above.)

'mds' is running, but you've erased the index and you've disabled indexing (which means it wont be trying to rebuild the index right now.)

*At this point* your 'mds' process should not be gobbling up extraordinary amounts of CPU or memory. If it is, then +something else+ is wrong beyond just index corruption.

Finally, re-enable indexing by typing this:
sudo mdutil -a -i on

Check the Activity Monitor again. At this point there should be a few md... processes that are quite active (because the system is rebuilding your indexes, so there is quite a bit of work being done.) This could take a few hours. Once it's finished, check Activity Monitor one last time to see if 'mds' is behaving better.

Message was edited by: Tim Campbell1

BobHarris wrote:

There is a more brute-force way to fix the problem involving the "Archive Install"

I do not think Snow Leopard has an Archive & Install option any more (which is a shame).

Ugggg <facepalm> Yep, I just verified. To my astonishment, they've removed the "Archive & Install".

If you have a good backup, you can effectively achieve the same result but via more steps and a longer amount of time.

hwpienaar wrote:
Tim.

After doing part 1, I got my RAM back and the problem stopped.

I am a bit confused with Part 2...?:

After I have opened the terminal you said: "...to rebuild your indexes, eject / remove any external volumes so that the only disk available on your Mac is the boot disk" What do you mean and how do I do this?

I didn't know if you happened to connect any external disks (e.g. USB drives, Firewire drives, etc.). If you did have any external drives attached, you can disconnect them as you normally would prior to typing those commands in part 2. That way any indexed volumes on the external disks wont be affected (spotlight indexes everything... if you connect an external disk, it'll start to build an index for it.) It's not critical... it was really just to speed things along. That way when you get to the part where you eventually rebuild the indexes again the computer will only need to rebuild the index for your internal drive and not worry about any external drives having their indexes as well. If you don't have any external disks attached then no need to worry about it.

It's good news that the problem went away when you shut down 'mds' in part 1. That means we've isolated the issue to mds. Normally this would be due to corruption in the indexes and rebuilding them would solve the problem.

hwpienaar wrote:
I tried "sudo mdutil -E -a" again and this time it gave me this:
Password:
Spotlight server is disabled.

Then I did the next step: "sudo mdutil -a -i" and it gave this:
Spotlight server is disabled.

No changes in mds or RAM - problem still remains and the spotlight server is disabled at the moment.

"Spotlight server is disabled." is the message it will give if the 'mds' process is not running when you type the "mdutil" commands. 'mds' needs to be running when you perform the 'mdutil' commands. If you had only just started it, then it may not have had enough time to startup to the point where it was ready to accept commands. Your Spotlight database is obviously not very happy, so you'll need to be patient. You want to make sure to give enough time after each command for it to complete your instructions. If all goes well, your patience will be rewarded and you will have a healthy computer once again.

Enable 'mds' by typing:

sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

Wait a minute or so for 'mds' to start. Since your Spotlight data is corrupt, this make take a few minutes for it to be ready to accept commands (hence the message about the state changing in one of your earlier threads -- that's something that normally happens so fast you don't even notice it, yet yours is taking a while to complete.)

Then try the command to erase the index again:

sudo mdutil -E -a

Since it seems to be taking a few minutes to complete things in it's corrupt state, I would wait a while to make sure it can complete the operation. You can check it's status with the "-s" flag. Example:

mdutil -s -a

(note that I didn't bother to prefix this with "sudo" -- since this is just a status command and is not actually trying to make any changes, you don't have to type "sudo" in front of the command. "sudo" is only used when you're trying to type a command that requires escalated privileges.)

When you're sure it's finished erasing the index, then you can re-enabling indexing:

sudo mdutil -a -i on

As per usual, it will take a while to index the entire contents of your hard drive.

My guess, based on the error messages you were seeing, is that your 'mds' process was taking too long to erase the indexes when you tried this before and the indexes didn't really successfully get purged.

Message was edited by: Tim Campbell1

hwpienaar wrote:
Hi Tim.
This is what I did so far regarding your last post. There are no changes regarding the problem so far, but I do see that the indexes are being re-build.

vc-41-2-61-42:~ henniepienaar$ sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist
Password:
com.apple.metadata.mds: Already loaded

Ok - so it's already running. That's good!

vc-41-2-61-42:~ henniepienaar$ sudo mdutil -E -a
/Volumes/Vodafone:
Indexing disabled.
/:
Indexing disabled.
/Users/henniepienaar:
Error: Index is already changing state. Please try again in a moment.

From this I can see that you have 3 indexes... it looks like you're indexing

1) an external hard drive named "/Volumes/Vodafone" (probably you didn't need to rebuild that index.)

2) the internal Macintosh hard drive (that's the "/") -- which might be the a problem index

3) it looks like your home directory has it's own index ("/Users/henniepienaar"). This normally doesn't get it's own index. Are you running FileVault or is this located in a separate partition (not as part of "/" as it normally would be)? I'm very suspicious of this index as this is the one that gives the message about wanting you to try again in a minute.

vc-41-2-61-42:~ henniepienaar$ mdutil -s -a
Spotlight server is disabled.
vc-41-2-61-42:~ henniepienaar$ sudo mdutil -a -i on
Password:
/:
Indexing enabled.
/Users/henniepienaar:
Error: Index is already changing state. Please try again in a moment.

Let the indexing process run for a while (at least a several hours or perhaps overnight) and check the status of the indexes again via "mdutil -s -a". Does it eventually stabilize so that you no longer get the "Index is already changing state" message?

If you are NOT using "FileVault" on your Mac (the encrypted home directory option) then you can ignore this next part. I'm only including it because I noticed your home directory has it's own index and normally a home directory would not have it's own index.

If you are running FileVault, this introduces a whole new issue as to whether there are any problems with your sparse bundle which are causing Spotlight some difficulty. If you do run FileVault you should definitely be keeping good backups (if your FileVaulted sparse-bundle is ever corrupted it's conceivable that you could lose all of your data. Nobody should use FileVault unless they are also running routine backups.)

If you are using FileVault and you have made a recent backup, you might want to rebuild the FileVault. The easiest way to do this is to turn FileVault off (it will tell you that you need to log out, then while you are logged out it will build a normal (non-FileVault) home directory and, once everything is copied out, it will delete the FileVault sparsebundle. You can then log back in, go to the Security options of System Preferences and turn FileVault back on again (which will prompt you to log out again... then spend a while building a brand new FileVault sparsebundle image. But since it's freshly created you wouldn't have to worry about it having any corruption issues.) The purpose of the backup is that if your sparsebundle already was corrupt it might not be able to succeed in copying everything without a problem... hence you need a good backup.)

Try disabling Spotlight and Time Machine.

Hi Tom;

Since the OP admitted that his Mac ran quickly in the new account, I have serious doubts how much good turning off Spotlight and Time Machine will have for him.

Allan

Allan Eckert wrote:
Hi Tom;

Since the OP admitted that his Mac ran quickly in the new account, I have serious doubts how much good turning off Spotlight and Time Machine will have for him.

This is why I asked about FileVault. I thought it was very odd that 'mds' (a system process) would be gobbling all the memory and CPU time -- but only for one user. When the OP posted his results from the mdutil command I noticed there was a Spotlight index specific to his home directory, and that is the index that seems to lag when he tells Spotlight to turn indexing on or off, etc.

There wouldn't normally be a Spotlight index specific to a user's home directory (it would normally just all be indexed under the index for "/" (the root volume). It would have it's own index if it were on a separate disk or at least appeared to be on a separate disk. This would be the case if he uses FileVault (since the sparse bundle is mounted as it's own volume and has it's own index.) Also, a FileVault is specific to each user -- so one user could have the problem while another user (since it's technically a separate FileVault) would be fine.

If the index specific to the user's home directory can't be rebuilt via mdutil (which would be the case if the sparse bundle itself has a problem), it's probably best to backup the sparse bundle, then cause it to rebuild by turning it off, then back on.