Apple Event: May 7th at 7 am PT

Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

Spotlight search keeps reindexing and not working

Hi there:

Long story short; Any idea at all why spotlight keeps reindeing, crashes and never finishes?

Background & Symptom:

Since upgrading to Mavericks my Spotlight haven't been indexing and searching properly. My main issue is Mail App because I search there 30-50 times a day but I can see the symptom is also there with searching for Apps or files. I've tried absolutely everything, been Googling and been on chat and phone support with Apple (still have time left on my extra warranty) at least 20 times and must say its shown no results so far.

Detail:

I've got 2 drives in it - 1 HDD and 1 SSD - everything OS, Apps and Mail is on the main SSD drive and I've even tried to have spotlight just index that but to no avail.

It worked prior to the upgrade from Mountain Lion to Mavericks.

I can see in console that my mds_stores crashes constantly (which maybe explains the behaviour) - maybe ~20 times a day.

The main thread that crashes is "com.apple.metadata.spotlightindex" but there are a couple of others too rarely (MDSSecureStore and com.apple.libdispatch-manager)

The other thing they all have in common is this message:

Exception Type: EXC_BAD_ACCESS (SIGSEGV)

and 50% of the mds_stores crashes have this in common:

Exception Codes: KERN_INVALID_ADDRESS at 0x0000000100000001

Here's some of the things I've done (and some of them many times in various orders):

- Reinstalled OS X Mavericks bare and reinstall/repaired on top of the existing install

- Moved a lot of the IMAP mails into local folders

- Tried different user account

- Uninstalled all kinds of stuff on the machine in case anything of that was the culrit

- Spotlight reindexing (both by dragging my drive into the spotlight exclusion as well as via the command line mdutil commands etc)

- Turned off FileVault (had to in order to get into Safe Mode)

- Reset PRAM (Cmd+Option+R+P)

- Have repaired disk permissions

- Have rebooted in safe mode

- Have rebooted in Recovery console and run resetpasswd/resetpassword and fixed ACLs

- Tried disabling everything but my mails in spotlight (Mail App)

- Lots of other things...

Any ideas at all?? I had to write here because I've rn out of options and I have on faith that Apple can sort it out (its been more than a month of open support cases).

Thanks a lot even for any ideas you might have!!

Cheers,

Daniel

MacBook Pro, OS X Mavericks (10.9)

Posted on Dec 9, 2013 4:22 AM

39 replies

Jan 8, 2014 3:49 PM in response to Barney-15E

Hi there,

Thanks for the comment. What are you asking me to do though? find "mdworker" in all the log files? (not sure what the mdutil command is for in this case and what digits to paste in as argument). FYI only the "powerstats" log file (looked only under /Library/Logs) contains the word "mdworker" (see below).

I did stumble onto something from your suggestion though.. my "powerstats" process crashes too (only really that one and the mds_stores - ocasionally another one or two but 85% mds_stores and 15% powerstats).

Thanks in advance for any further comments, insights, suggestions etc..

Cheers,

Daniel

Barney-15E

Level 10

112,776 points

Jan 8, 2014 3:56 PM in response to Daniel_iversen

In Console, select All Messages.

In the search field, type mdworker

Look at any results. They will likely be one line with a black triangle on the left. Click the Triangle and you will see the rest of the message.

In that message, it will display that command (with the digits filled out) that you can copy and type into Terminal to find the file that is causing the issue.

I don't know if this will solve the problem, but it might.

Powerstats isn't crashing. Its just a diagnostic log that runs once per day. It also seems to have something to do with mdworker.

Jan 9, 2014 2:36 AM in response to Barney-15E

Yesterday I set my "~/Library" folder to Private in Spotlight Settings. That stopped the reindexing, but also my Mail from being searchable.

Today I tried to search the console for "mdutil -t" messages, and plenty were found in "~/Library/Application Support". Mostly iWork and SyncServices files. I will try to limit the spotlight privacy to Application Support and see what happens.

Thank you!

Jan 13, 2014 1:48 PM in response to Daniel_iversen

I think I may have resolved this, but fingers crossed and lets see (18 hours and counting)..

Basically I went into spotlight and excluded everything except for a couple of mailboxes and/or folders inside ~/Library/Mail/V2 and it calmed everything down. Its hard to find out what exactly caused it because I did this a while back and the symptom happened shortly (~ day) after again.

I'm guessing maybe there are some corrupt mails bu not sure. Will update the post later on so others can hopefully get some benefits out of this as well.. Its only been 3 or so months without proper search 😟 I suspect that Mavericks Mail App or Spotlight is a piece of crap - were is the defensive coding that prevents the whole index to blow up!!?

Jan 13, 2014 1:56 PM in response to Daniel_iversen

Nevermind, this piece of poo Spotlight has stopped working again 😟..

Jan 13, 2014 2:03 PM in response to Henrik Dalgaard

Well the problem is back. It was not something in Application Support.

I fear it is in Mail somewhere. This is where I use Spotlight the most.

Jan 14, 2014 3:15 PM in response to Barney-15E

None of the messages in the console had the triangle and mdutil commands under it (there were only a couple of messages that could be expanded via the triangle icon).

Any other ideas anyone?

Thanks,

Daniel

NB: Has anyone resulted to trying to use a non-Spotlight-based desktop search tool of any kind and what is the experiences?

Jan 23, 2014 2:17 AM in response to Daniel_iversen

Hi Daniel,

I've got the exact same problem. Also happened after upgrading to Mavericks and it's driving me nuts. My Spotlight starts reindexing my entire system on every reboot, which takes about an hour every time. I've also tried almost everything you listed (resetting Spotlight via the UI, via the terminal, etc.), nothing helps. I really only need Spotlight to be able to search my mail, but excluding everything in my homedrive except ~/Library/Mail doesn't solve my problem either.... It still starts reindexing on every reboot, and not only my mail dir, but also the other dirs that I did not exclude (I checked by watching the files opened by the index process: "sudo fs_usage -w -f filesys mdworker")

So I've now excluded my entire home-dir, so I at least can start apps via Spotlight. But I can't search my mail anymore, which is a real pain.

If there's anything I can do (e.g. add info to your support case), please let me know.

Furthermore, how did you get the info on the crashes of the Spotlight index processes? I went through my old system.log files, and the Console app, but I don't really have crashing Spotlight processes as far as I can tell.

Cheers,

Duncan

Jan 23, 2014 2:32 AM in response to Daniel_iversen

Now I have excluded my ~/Library/Mail from Spotlight search. No reindexing has happened for the last 24 hours.

I have the same problem on two Macs, where I use the same IMAP accounts. I wonder if Spotlight does something special for email. Perhaps 150.000 messages may be too much for Spotlight. Time for a binary search.

Jan 23, 2014 2:43 AM in response to Henrik Dalgaard

I'm gonna try that. I also have an IMAP account configured (corporate mail) with about 5GB of e-mails .....

Jan 23, 2014 3:20 AM in response to Duncan Doyle

Hi there Duncan,

I've incidentally got a few Apple Support cases for this issue over the time (I think the system created a new one every time) but the last one with the most data, history, info and experiments is 541648142 and it was created mid Deecmber so you could refer to that.

The spotlight crashes was in my console it is the mds_stores process that keeps crashing.

I think Microsoft Outlook for Mac doesn't have this issue (maybe its more robust and transactional as well and doesn't stall as much as Mail App either) - would be worth a test I think. But it would be pretty bad if we had to buy that just because Apple's stuff doesn't work (and I'd think MS Outlook wouldn't be clever with PowerNap mail fetching either which was something I was looking forward to using in the future on a newer machine)..

If it does turn out that Microsoft Office Outlook for Mac is the most robust for mails then here are cheap sourcing options (depending on what you qualify for);

- Academic license for students: http://www.microsoft.com/student/discounts/itsnotcheating/default.aspx

- Home User Program (HUP): http://www.microsofthup.com/

- BizSpark (for startups): http://www.microsoft.com/bizspark/

- Amazon: http://www.amazon.com/s/ref=nb_sb_noss_2?url=search-alias%3Dsoftware&field-keywo rds=microsoft+office+2011+for+mac+%22home+and+business%22&rh=n%3A229534%2Ck%3Ami crosoft+office+2011+for+mac+%22home+and+business%22

Anyone else have any thoughts or ideas on how we can fix this Spotlight / Mail App issue/symptom?

Thanks in advance.

Cheers,

Daniel

Jan 23, 2014 5:54 AM in response to Daniel_iversen

Wonder if the upcoming patch OS X Mavericks 10.9.2 Build 13C39 fixes this, any ideas?

Jan 23, 2014 11:00 AM in response to Daniel_iversen

I have filed it to Apple as a bug

Bug ID #: 15890594

Bug Title: Spotlight reindexes my Mac on a daily basis

They asked me to perform a diagnostic report with the command

sudo mddiagnose

in the Terminal.

That made a 20 MB file I have attached to the bug report.

I hope this will help to get the bug hunted down!

Jan 23, 2014 11:12 AM in response to Henrik Dalgaard

Det lyder godt Henrik!

Let's see what comes out of that, fingers crossed!! Funny that over my 3 months of working with Apple Support on this and lots of reinstalls and experiments and data captures I was never asked to run "mddiagnose"! :-|

Lets see. In the meantime, anyone that has any updates for the rest of us (or even ideas) that would be great to hear.

Cheers,

Daniel

Jan 23, 2014 11:41 AM in response to Daniel_iversen

Håber Apple får spor nok at gå efter!

Apple has heard about it before in bug #15838350

Now I have sent them one more report from my other Mac as well. After a Restart from the Apple menu Mail triggered the reindix.

So far Apple has fixed several of the bugs I have reported. Mostly bugs which more people have found and told them about - do if a bug report gets the "Duplicate" stamp it is a good sign.

Cheers,

Henrik

Spotlight search keeps reindexing and not working