SpotLight Indexing Problems

Question

Level 1

15 points

SpotLight Indexing Problems

Some insights in the Spotlight Indexing problems where indexing takes forever. It's my gut feeling that this is caused by some files that indexing is choking on. To check on this open the console and select Console Messages. Start indexing. After a few minutes you'll observe the following symptoms. The Console will throw this error message every few seconds

3/13/08 3/13/08 8:43: 35AM com.apple.launchd[1] (0x10aa60.translate[608]) Exited abnormally: Bus error
Disk activity will slow down or stop
CPU activity will drop to 1 or 2%
Spotlight Indexing Estimate time will start to increase.

If you let this run the estimated time will increase and decrease indicating that it is making slow progress but will still indicate a time of over 100 hours. Watch the console log and you'll also see the time between the error messages will increase during the decrease in time.

Interestingly I had exactly the same problem and error messages when I first tried to set up Time machine. I solved that by the deletion of some very old files.

I have temporarily solved my problem by excluding the three library folders from the indexing. Indexing then took about 15 minutes. It did throw a few of those error messages but seemed to get by what ever the problem was.

My question is there any way to determine what file is causing the problem?

Mac Pro Early 2008, Mac OS X (10.5.2)

Posted on Mar 13, 2008 9:21 AM

Reply

Answer 1

dechamp

Level 4

3,495 points

Mar 13, 2008 9:38 AM in response to Mike Riordan

Not many clues here.

Internal or external drive?

What directories did you have to exclude, and what kind of files?

Can the directories be copied to another drive or is there some directory damage or file corruption?

Where did the files come from? What are the file's permissions?

Have you repaired permissions by using Disk Utility?

Sorry, more questions than answers...

Reply

Answer 2

Mike Riordan Author

Level 1

15 points

Mar 13, 2008 9:59 AM in response to dechamp

It's the internal System Hard Drive.

The Folders excluded are:
HD/Library
HD/System/Library
HD/User/Username/Library

There is no file damage nor Corruption in these folders.

The folders/files were part of the Leopard installation. However, I suspect some maybe leftovers from previous systems even though I did an Archive and Install.

All permissions have been repaired.

Reply

Answer 3

dechamp

Level 4

3,495 points

Mar 13, 2008 11:41 AM in response to Mike Riordan

Those particular folders are problematic for Spotlight, and I don't have the answer for you. Spotlight is supposed to not index parts of it, but still index your mails, addressbook and some other data. Not a lot of documentation laying around for the exact details...

You might try to re-index the entire drive by adding it to the Privacy option and then removing it again and the other folders you've excluded.

Spotlight will start the reindex automatically a short while after you close the preferences window. Be prepared for an extended time for the re-index to be accomplished. Just let it work through completely.

Reply

Answer 4

Mike Riordan Author

Level 1

15 points

Mar 13, 2008 12:48 PM in response to dechamp

Actually, that's what I did that started the problem. I found that that my drive was only partially index so I added the drive to the privacy option and removed it. After about 15 minutes it stated to throw those errors and the estimated time went from 20 minutes to 125 hours and stayed there. I launched the console and saw the errors every 5 sec. I added the HD to the privacy option to stop indexing. Finally I removed the HD from privacy and added the Libraries. It took about 90 minutes to completely index the HD. I have about 90 GHz on the HD. I suspect that those who are having extend indexing time are receiving those errors.

What I find most interesting is that when I first tried to use the Time Machine I had exactly the same symptoms and exactly the same errors.

Reply

Answer 5

dechamp

Level 4

3,495 points

Mar 13, 2008 2:05 PM in response to Mike Riordan

Yeah, I figured that might have been what happened. But anyway, if you continue to block Spotlight's access to the user library files are you still able to use Spotlight to search your emails and addressbook entries?

I would add a temporary contact to the AddressBook, quit the program and then see if it comes up in Spotlight. I'd also look at the text of a very recent email and try to find some word or phrase that's distinctive and see if it gets found by Spotlight.

Those are probably the most common Spotlight searches I do when clients call, so for me, that ability is imperative. YMMV

Reply

Answer 6

Mike Riordan Author

Level 1

15 points

Mar 13, 2008 3:18 PM in response to dechamp

In as much as I have never had a complete index of the HD non of the address book nor mail is searchable via spotlight. The next step is to find what in the User/Library is causing the problem. I'll take half of the folders in Library and let Spotlight index them. If I run into a problem then it's in that half and I'll split them in half and so on until I narrow it down to a single folder.

The Error message generates a report in Crash Reporter that if it happens in another program it will crash the program. Not in Spotlight nor Time Machine. It just keeps repeating until it gets over the rough spot and then goes on.

Reply

Answer 7

Mike Riordan Author

Level 1

15 points

Mar 14, 2008 9:22 AM in response to Mike Riordan

I've gotten it narrowed down to what I consider the essentials.

1. I removed the HD/Users/Username/Library from the Spotlight Privacy List. Spotlight then indexed that library with no errors. This added Mail, Address Book and others to the Spotlight Index giving me access to the find data.

2. I changed HD/System/Library privacy to HD/System/Library/Frameworks in the Spotlight Privacy option (Delete and Add).

3. I had to leave the HD/Library in the Spotlight Privacy Option alone. When I tried to change it, it started to throwing errors.

I could probably narrow it down further but I consider these exclusions acceptable as it's unlikely that I'll ever search for these items.

Note--When you make these changes Spotlight does not tell you it's indexing but you will notice Disk Drive activity and increased CPU usage for a few minutes the additional data is indexed. You should have the Console open and set to Console Messages to monitor for errors.

I consider the problem resolved.

Reply