Leopard's Spotlight doesn't search Mail messages

Hi Imran,

a- I agree that different people have different experiences. That's why I said, "That was not the case for me," when you said, "I have always felt it was a permissions issue somewhere."

b- I run as root (yes, I know there are risks).

c- The way I solved the problem is stated in the thread in message ID=6839305 where I said, "After completely erasing my HD and restoring from my Time Machine backup, Mail searches started working again. Whatever was broken, was fixed by this."

Hopefully this will be helpful to others.

Deleting the Spotlight index and reindexing didn’t work for me -- twice. Putting the Mail folder in and out of the Spotlight Privacy preferences setting never worked either -- numerous times (including waiting a while before removing it, and even added and removed the whole Library folder).

Some others have reported an "Archive and Install" of the Mac OS solved their Mail search problems. Not sure it works for everybody, though, but it may be one of the better solutions. So, I did an Archive and Install of Leopard, 10.5 -- using the discs that came with my current model iMac. Immediately after the installation and a restart, I tried Mail's search and Spotlight's search and it found Mail messages again, instantly. Other expected files were found in Spotlight also, so it appears that no "reindexing" of Spotlight was necessary after the re-install. And it seemed like all the Mail messages were probably already indexed -- since I didn't notice any index activity of Mail. I did the search immediatlely after the re-install. So, it may be that the missing Mail messages were already indexed, but there's is (was) something preventing Mail and Spotlight from finding them, or displaying them.

I went on to re-update to current 10.5.2 and re-updated some other Apple files using Software Update... like Quicktime, etc. After the updating, Mail & Spotlight search continues to work as it's supposed to. To be more specific, considering some limited successes reported with various other solutions, after my Archive and Install I can now search keywords to find Mail messages in Entire Message of All Mailboxes, or any specific mailbox. And new received and sent emails are immediately searchable. Of course, the From, To and Subject searches also work.

My Archive and Install, including the updates, took around 3 hours, I think. But there's no need to sit there and watch the progress bars all the time, you can do other things. After re-installing Leopard I had to reinstall Dreamweaver, and probably will some other things, but most everything else just seems to work like I left it. Some applications say, "You are opening for the first time" when first opening them after the re-install. That kind of thing is to be expected after a re-install.

Apple details on reinstalling: http://docs.info.apple.com/article.html?artnum=301270

I've been super frustrated with this issue as well, and I just solved this problem myself. The problem was the computer not having an index of the body text of the e-mail that I brought over from the old machine (which was running 10.3.9).
It looks like some people have tried pretty crazy things. I found a solution that is 100% manual and safe. Depending on how numerous and big your mailboxes are, it should go fairly quickly. My solution is this:

I created a new mailbox, and named it ZZZ. Then I copied the entire content of a particular folder into ZZZ (leaving the original folder empty). Then I copied all the e-mail from ZZZ back into the original folder. I repeated this process for every mailbox, and now I have what I think are full indexes of each mailbox.
This took me only about 20 minutes to do this move and move-back for every mailbox folder. Not bad. I hope this helps.

Here's gow I fixed it...

Be aware I use imap mail, so all my messages are always on the server!

I simply deleted the contents of /user/library/mail/ and started mail up.
It then proceeded to download all the email from my imap server again, (the account details were still there).

After however long that took it all searched entire fine again.

I assume that something happens during the initial file vault encryption to existing email folders.

Remember, if you have regular pop mail where the messages reside on your computer only you will need to take steps to backup then somehow restore them.

Richard.

<Edited by Moderator>

Spotlight is a complete pile of crap ... we need to spread the word about this to put pressure on Apple to fix this. I get LOST of feedback on this - its a plague to an unbelievable amount of users!

Anyways - stopping all indexing, doing a sudo rm -R of the .Spotlight-V100 and starting spotlight again sort of works - for some time ... Im trying now to exclude every folder containing fonts - my log was filled with font validaton errors - this is completely insane - Leopards font handling is completely broken as well ...

What does apple have to say about this? Ever since spotlight was introduced, I can NEVER trust my mail searches. I NEVER find what I KNOW IS THERE!

This is the most fundamental function of a mail client. I can't do my job, because I can't find emails that are necessary for me to do my job. This is outrageous, and I have no recourse to follow.
This mdimport CRAP is a STUPID WORKAROUND.

*APPLE: TELL ME YOU ARE GOING TO FIX THIS PROBLEM.* TELL ME YOU CARE!

<post edited by host>

I have found that Spotlight actually does an excellent job, you just have to get it working. If you read my post (5 or so above) you can see how to get Spotlight to search your mailboxes. If you have a different issue, then I'm sorry I can't help you. But I'd suggest try what I say above.

Hi dreamburn, welcome to the Discussions.

In order for Apple to know about the problem, you need to tell Apple, not these Discussions. These Discussions are for users to help other users. To tell Apple, please use the "Provide Mail Feedback" item in the Mail menu. That will send it to Apple.

Just wanted to give my 2 cents. I had this problem too with my IMAP email. After lots of investigation I narrowed the problem down to one thing. .imapmbox folders. The content was actually indexed. It just would not show any results from a folder that had .imapmbox as a folder extension name. as soon as a renamed the folder to something else spotlight would show me results from within. I never figured out what was causing spotlight to do this. I wound up reformatting my drive and starting from scratch. No problems since the clean setup for about two weeks now.

+What does apple have to say about this? Ever since spotlight was introduced, I can NEVER trust my mail searches. I NEVER find what I KNOW IS THERE!+

+This is the most fundamental function of a mail client. I can't do my job, because I can't find emails that are necessary for me to do my job. This is outrageous, and I have no recourse to follow.+
+This mdimport CRAP is a STUPID WORKAROUND.+

+APPLE: TELL ME YOU ARE GOING TO FIX THIS PROBLEM. TELL ME YOU CARE!+

••••

This was a very gnarly problem for me ( 14 gigs of mail ) and bottom line it's not really Apple's fault.
It's a legacy problem.

*The ONLY solution that has been reliable and complete is to install a fresh Leopard, bring it up to date then use migration assistant to pull from the previous system* that would not search properly.

Let Spotlight do it's new indexing and you should have no further problems. Leopard has a new mail structure which makes it much faster but often causes issues when it's been upgraded. The import gets rid of that by replacing the structural issues and just pulling in the individual mail files and leaving the flawed structure behind.

Rebuilding mailboxes, not letting message load per mailbox go above 5,000 per and running something like Onyx regularly also are important.

One undiscussed aspect is drive optimization which has an impact on large mail file structures. iDefrag does an excellent job.
While X does a decent job of avoiding fragmentation of individual files it does nothing to keep critical file structures closely grouped ( optimized for access ).

•••

I note David Miller took a similar approach with a fresh start and no further problems.



+Re: Leopard's Spotlight doesn't search Mail messages+
+Posted: Apr 8, 2008 11:14 PM+

+Just wanted to give my 2 cents. I had this problem too with my IMAP email. After lots of investigation I narrowed the problem down to one thing. .imapmbox folders. The content was actually indexed. It just would not show any results from a folder that had .imapmbox as a folder extension name. as soon as a renamed the folder to something else spotlight would show me results from within. I never figured out what was causing spotlight to do this. I wound up reformatting my drive and starting from scratch. No problems since the clean setup for about two weeks now.+

There is a big variety of apparently related problems for Apple to deal with. The solution of my problem of not searching on "Entire Message" is in another thread in this forum:
http://discussions.apple.com/thread.jspa?messageID=6968767&#6968767

"Entire Message Search Won't Work"

Using three different sets of Terminal commands should have worked, but did not because a bug failed to deal with an old font suitcase stashed in my user directory. Apple Care helped me out with Archive and Install of Leopard on my aluminum iMac.

Excellent work!!! Thanks for saving me from lots of headaches.
Your solution works like a charm.
- dabcap

Hello,

Try to install the new updates of mac os x (10.5.3) and spotlight search perfectly all mails.

best regards,

Leopard Mail Entire Message Search - FIXED!!!

My Original Post is here:
http://discussions.apple.com/thread.jspa?messageID=7462907#7462907

Posted by: DaveC
This is a known issue. Check out:
http://www.betalogue.com/2008/02/25/mac-os-x-105-tip-repairing-broken-spotlight- searches-in-mail/
http://www.betalogue.com/2008/03/28/mac-os-x-105-broken-spotlight-indexing-for-m ail-messages-caused-by-presence-of-older-mail-application/

Thanks Dave. I went to Betalogue and went through all the posts and I finally found a way to fix the Search Entire Message Spotlight problem in Apple Mail - at least it worked for me. I have been trying to fix this with all sorts of solutions for months and it turns out it was related to a problem with the Launch Services Database. I confirmed this by looking inside my user/library/mail folder and seeing that inside each Mailbox folder, the sub mailboxes all had standard folder icons and not the inbox tray style icons.

Note that you can easily fix this through Terminal Commands, and not all the following steps are completely necessary, but there are many people looking at these threads that want to just push buttons...

*== So here are my simple steps to fixing Apple Mail Search ==*

1. Attach and turn on any external hard drives you have that may have a copy of Mail.app on them

2. Delete all OLD copies of Mail.app that you can find (don't worry about Time Machine), but make sure you keep the latest version of Mail.app in your working Applications folder.
~ This is so that the LaunchServices database won't easily become corrupt again

3. Install the latest version of Onyx - free to use and download from www.macupdate.com
~ I suggest Onyx, because some people don't want to mess with the Terminal.

4. Quit all open applications and run Onyx (it will want to check your Hard Drive etc...that's OK).

5. Go to the Maintenance pane and then the Rebuild tab and select only the LaunchServices box. Click Execute.
~ This should fix the primary problem with searches

6. When that process finishes, select only the Mail's Envelope Index box. Click Execute.
~ This should fix related problems with other searches

7. When that process finishes, select only the Spotlight Index box. Click Execute. (This process of Rebuilding your Spotlight Index could take hours depending on how big your hard drive is).
~ This will give you a clean fast Spotlight database that Mail uses for Entire Message searches.

8. Quit Onyx

9. Open Mail.app and test your Entire Message Search!

====

Hope that helps - it certainly, finally, did for me!

Bryce.