mds repeatedly crashes, can't build Spotlight index

Question

Level 1

0 points

mds repeatedly crashes, can't build Spotlight index

Hello,
I upgraded to Snow Leopard a few days ago and since then my machine is locked in an attempt to build the Spotlight index. First I thought this would just take a while and that I should be patient. But then I had a closer look at Console and found that mds repeatedly crashes and then starts all over again.

Below is an excerpt from the Console (I deleted some reoccurring messages to keep it short).
I started to notice the mds activity after I installed XCode 3.2.1 but that might be just a coincidence.

Also, out of curiousity, why is there an mdworker and an mdworker32 process?

Thanks much in advance for any insight,
schnapp

...
Feb 19 16:13:18 preon com.apple.launchd[1] (com.apple.metadata.mds[973]): Job appears to have crashed: Segmentation fault
Feb 19 16:13:18 preon mds[987]: (/)(Error) IndexGeneral in SIOpenIndex:check crashstate: 1 for /.Spotlight-V100/Store-V1/Stores/D599D2A2-79F2-4E04-90CC-1B203B60771D
Feb 19 16:13:18 preon mds[987]: (Normal) DiskStore: Rebuilding index for /
Feb 19 16:13:18 preon mds[987]: (Normal) DiskStore: Creating index for /
...
Feb 19 16:21:14 preon mdworker32[1014]: kCGErrorFailure: Set a breakpoint @ CGErrorBreakpoint() to catch errors as they are logged.
Feb 19 16:27:46 preon /System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework /Versions/A/Support/mdworker[1002]: zip importer encountered an error (2) scanning "/Users/glui/Library/Mail Downloads/Beschreibung doc.zip".
...
Feb 19 16:31:28 preon [0x0-0xa60a6].com.apple.ActivityMonitor[1032]: Activity Monitor: OpenScripting.framework - scripting addition "/Library/ScriptingAdditions/QXPScriptingAdditions.osax" declares no loadable handlers.
...
Feb 19 16:33:16 preon mdworker32[1027]: NSDataMessageStore: couldn't find body
Feb 19 16:33:37: --- last message repeated 565 times ---
Feb 19 16:33:37 preon mdworker32[1027]: * process 1027 exceeded 500 log message per second limit - remaining messages this second discarded *
...
Feb 19 16:45:49 preon Safari[268]: Periodic CFURLCache Insert stats (iters: 362) - Tx time:0.008139, # of Inserts: 5, # of bytes written: 34679, Did shrink: NO, Size of cache-file: 182648832, Num of Failures: 0
Feb 19 16:49:22 preon Safari[268]: INSERT-HANG-DETECTED: Tx time:8.096443, # of Inserts: 0, # of bytes written: 0, Did shrink: YES
...
Feb 19 16:50:38 preon mdworker32[1027]: Unable to parse date ( )
Feb 19 16:53:29 preon mdworker32[1027]: * Failed to decode data, treating as binary
Feb 19 16:54:00 preon mdworker32[1027]: NSDataMessageStore: couldn't find body
Feb 19 16:54:30: --- last message repeated 41 times ---
Feb 19 16:55:09 preon mds[987]: /SourceCache/Spotlight/Spotlight-507.4/index/ContentIndex/CIPositions.h:215: failed assertion 'len == 0 || bytes[len-1]==0' /.Spotlight-V100/Store-V1/Stores/D599D2A2-79F2-4E04-90CC-1B203B60771D (hfs, t: 0x11, st: 0x1, f: 0x480d000)
Feb 19 16:55:11 preon com.apple.ReportCrash.Root[1255]: 2010-02-19 16:55:11.248 ReportCrash[1255:2903] Saved crash report for mds[987] version ??? (???) to /Library/Logs/DiagnosticReports/mds 2010-02-19-165511localhost.crash
Feb 19 16:55:11 preon com.apple.launchd[1] (com.apple.metadata.mds[987]): Job appears to have crashed: Segmentation fault

(and here the cycle starts again)

MBP (early 2008), Mac OS X (10.6.2)

Posted on Feb 19, 2010 8:24 AM

Reply

Answer 1

Best reply

Kappy

Level 10

361,741 points

Feb 19, 2010 9:03 AM in response to schnapp

Could be a corrupted index file. Visit The XLab FAQs and read the FAQ on Spotlight issues.

Reply

Answer 2

schnapp Author

Level 1

0 points

Feb 19, 2010 9:31 AM in response to Kappy

Thanks Kappy.
OK, the FAQ you linked to basically suggests to rebuild the index.
As I understand it you do so with the command: sudo mdutil -E /
When I performed this operation the indexing started OK, however I can't see how this is different from what the OS did anyway after mds crashed. In any case this time the crash happened even faster:

Feb 19 18:12:32 preon sudo[1414]: glui : TTY=ttys000 ; PWD=/Users/glui ; USER=root ; COMMAND=/usr/bin/mdutil -E /
Feb 19 18:12:33 preon mds[1396]: (Normal) DiskStore: Creating index for /
Friday, 19. February, 2010 18:17:37 Europe/Berlin
Feb 19 18:18:45 preon Safari[268]: Periodic CFURLCache Insert stats (iters: 389) - Tx time:0.710683, # of Inserts: 11, # of bytes written: 198520, Did shrink: NO, Size of cache-file: 182745088, Num of Failures: 1
Feb 19 18:20:15 preon mdworker32[1431]: kCGErrorFailure: Set a breakpoint @ CGErrorBreakpoint() to catch errors as they are logged.
Feb 19 18:20:23 preon com.apple.ReportCrash.Root[1434]: 2010-02-19 18:20:23.702 ReportCrash[1434:1407] Saved crash report for mds[1396] version ??? (???) to /Library/Logs/DiagnosticReports/mds 2010-02-19-182023localhost.crash
Feb 19 18:20:25 preon com.apple.launchd[1] (com.apple.metadata.mds[1396]): Job appears to have crashed: Segmentation fault
Feb 19 18:23:14 preon mds[1462]: (/)(Error) IndexGeneral in SIOpenIndex:check crashstate: 1 for /.Spotlight-V100/Store-V1/Stores/D599D2A2-79F2-4E04-90CC-1B203B60771D
Feb 19 18:23:14 preon mds[1462]: (Normal) DiskStore: Rebuilding index for /
Feb 19 18:23:14 preon mds[1462]: (Normal) DiskStore: Creating index for /

Below is the first chapter of the mds crash report. Let me know if it helps to post the whole thing:

Process: mds [1396]
Path: /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadat a.framework/Versions/A/Support/mds
Identifier: mds
Version: ??? (???)
Code Type: X86-64 (Native)
Parent Process: launchd [1]

Date/Time: 2010-02-19 18:20:20.495 +0100
OS Version: Mac OS X 10.6.2 (10C540)
Report Version: 6

Exception Type: EXC BADACCESS (SIGSEGV)
Exception Codes: KERN INVALIDADDRESS at 0x00000001129f1ff9
Crashed Thread: 5 Dispatch queue: Index Flush Scheduler for spindle 1

Reply

Answer 3

Kappy

Level 10

361,741 points

Feb 19, 2010 9:40 AM in response to schnapp

If you read through the FAQ you should find out how you can first completely remove the old index. I think that if the old index is sufficiently corrupted then reindexing may not be possible. However, the Spotlight related index files are invisible so you need to do some Terminal work to remove them. This is not really my area of expertise - Spotlight problems - but Dr. Smoke's FAQ should help if I remember right.

Sorry for not being more helpful.

Reply

Answer 4

schnapp Author

Level 1

0 points

Feb 19, 2010 11:44 AM in response to Kappy

Thanks again.
The way I understood the FAQ, the index only has to be removed manually under Tiger.
But I gave it a shot anyway.
Unfortunately it doesn't change the situation 😟

Reply

Answer 5

Kappy

Level 10

361,741 points

Feb 19, 2010 11:51 AM in response to schnapp

I can't help you further. Best I can suggest is an extreme measure - erase and reinstall OS X or erase the drives which have damaged indexes.

Reply

Answer 6

BDAqua

Level 10

239,105 points

Feb 27, 2010 4:01 PM in response to schnapp

The cure for that in several instances has been to Safe Boot, Repair Permissions, Archive & Install, (or apply the latest big Combo Update), Repair Permissions, reboot...

http://63.246.25.159/message.jspa?messageID=8747139

But saw it in Tiger also, especially when all those other fixes don't work, and Segmentation Fault is mentioned.

Reply

Answer 7

schnapp Author

Level 1

0 points

Mar 1, 2010 3:58 PM in response to BDAqua

Thanks much BDAqua
So far I tried your steps with the Combo Update as I was a bit wary about doing Archive&Install.
Sadly this didn't help.
I'll probably wait for 10.6.3, maybe more luck with that, who knows....
If that doesn't work then I'll try A&I

Reply

Answer 8

schnapp Author

Level 1

0 points

Mar 4, 2010 7:25 AM in response to schnapp

additional observation:
mds also crashes repeatedly when trying to index an external disk

Reply

Answer 9

Mar 4, 2010 9:43 AM in response to schnapp

Hello,

Try using the program "OnyX" to remove spotlight's index as well as clear system, user, and boot caches. This may take a while, but should hopefully reset spotlight completely. You may also benefit from running other maintenance routines that are available in OnyX.

http://www.titanium.free.fr/index_us.html

Reply

Answer 10

schnapp Author

Level 1

0 points

Mar 4, 2010 4:02 PM in response to Topher Kessler

Thanks Topher.

Yes, Onyx.
I tried that after I found this thread where someone had similar problems:
http://discussions.apple.com/thread.jspa?messageID=11109377

However, this didn't work for me 😟

Reply

Answer 11

schnapp Author

Level 1

0 points

Mar 4, 2010 4:22 PM in response to BDAqua

Now, regarding Archive&Install.
There seem to be 2 opinions on how to do that under Snow Leopard:
http://forums.macrumors.com/showthread.php?t=776924

#1 Snow Leopard does A&I by default
#2 no, you have to do a bit more:
http://intlect.com/how-to-archive-and-install-os-x-snow-leopard/

So I went for #1, however there's no "Previous System" to be found afterwards, so I guess it didn't really archive anything. And neither did it solve my problem.

But looking into #2 I fail to see how this is different from wiping my main disk, installing SL from scratch, and then copying all my data from a backup. Plus installing and registring all apps that can't be just moved over, something that I'd rather avoid.

I hope there's a #3

thanks

Reply

Answer 12

schnapp Author

Level 1

0 points

Mar 30, 2010 2:55 AM in response to schnapp

Just made my final attempt and tried everything (safe boot, repair permissions, rebuild index, Onyx cleanup, etc.) again, this time with the 10.6.3 Combo.
Still the same sh*t - three mds crashes within 5 minutes 😟

Will set up a fresh disk now, unless someone has new ideas what to try.
Or maybe I'll just go back to 10.5

Reply

Answer 13

Mar 31, 2010 5:05 AM in response to schnapp

Hi,

Same issue here, HFS compression was active on my Documents folder...

According to your system.log
mdworker1002: zip importer encountered an error (2) scanning "/Users/glui/Library/Mail Downloads/Beschreibung doc.zip".

Try to remove this file, it might be a protected zip that spotlight cannot index.

Verify if a compression utility like Squeeze has been used to activate HFS compression on some folders.
This could prevent Spotlight (mds) to index.
If so, uncompress the affected folders using the same utility then re-index.

Try to isolate the volumes by adding all the volumes to the Spotlight privacy list in system preferences, then wait until indexing is complete before removing another from the privacy list.

You can also use the terminal command mdutil to erase and rebuild the indexes
To display the manual, type man mdutil in a terminal window.

Best of luck

Reply

Answer 14

schnapp Author

Level 1

0 points

Apr 2, 2010 3:40 AM in response to loloshark

thanks much loloshark

Removing the single file didn't help (I actually tried that earlier w/o mentioning it here). When mds crashes now I don't get pointed to a particular file any longer.

Regarding HFS compression:
I found this OS X Hint:
http://www.macosxhints.com/article.php?story=20090902223042255
where one of the commenters provided a tool called afsctool which lets you examine the compression state of files and folders (and change it).
For now I examined three folders:
/System 127919 compressed files
/Applications 47573 compressed files
/Users 0 compressed files

So if HFS compression is the problem, then everything should work OK if I allow Spotlight to index /Users and nothing else, right? However, this wasn't the case 😟
In turn, I don't get an mds crash if I allow Spotlight to index the (heavily compressed) /System folder (and nothing else).

In any case I'll take your advice to try more isolating.
I'm only dealing with a single volume at the moment, but I'll use the privacy list to drill down into the file hierarchy to find the offending files.
(I'll tell you next year when I'm done with that 😉 )

Reply

Answer 15

schnapp Author

Level 1

0 points

Apr 2, 2010 4:40 PM in response to schnapp

latest update: the isolating experiments didn't yield any useful results. The whole behaviour is completely inconsistent, i.e. the same set of excluded folders (Privacy in System Preferences->Spotlight) sometimes works and sometimes not. So I figure something independent from the folder contents is disturbing the process. Maybe mds runs out of memory or chokes after a million files or whatever. Or maybe it's the mdimporter of some application that blows it up.
Can't believe how many hours and days I've wasted on this nightmare. I'll go for the fresh install now.

But thanks to everyone who tried to help!

Reply