Spotlight crashes regularly despite re-indexing

Question

Level 1

0 points

Spotlight crashes regularly despite re-indexing

rMBP late 2013, 2.6 GHz, 16 GB RAM, 512GB storage; bought in January 2014

OSX Mavericks 10.9.4

Since about a month or so my Spotlight crashes regularly. I have re-indexed several times, both by adding and removing the Mac HD in the Privacy tab in the Spotlight Preferences as well as from the terminal using:

--

sudo mdutil -E /

--

Each time re-indexing finishes without problems (within a few hours or so, I usually let it run overnight), Spotlight works smoothly for a couple of days, and then crashes again out of the blue with no apparent relation to specific tasks that I'm doing at the time. No mds tasks (which run full steam while indexing) show significant activity after this happens.

Interestingly when I attempt to re-index after Spotlight stops working, I get the following error in the Spotlight Preferences when I try to add Mac HD to the Privacy list:

--

Privacy List Error

The item couldn't be added or removed because of an unknown error.

--

and the (presumably corresponding) errors in the command line:

--

sudo mdutil -sv /

/:

Error: unexpected indexing state. kMDConfigSearchLevelTransitioning

sudo mdutil -i off /

/:

Error: Index is already changing state. Please try again in a moment.

sudo mdutil -i on /

/:

Error: Index is already changing state. Please try again in a moment.

--

Trying again in a moment doesn't change anything. The only "fix" is to restart the computer, re-index, and wait for Spotlight to crash yet again every few days. No other apps seem to be affected by this.

Any ideas on what could be causing this problem and how to fix it? I rely heavily on Spotlight so this problem has been very frustrating. Many thanks in advance for any help or advice.

MacBook Pro (Retina, 13-inch, Late 2013), OS X Mavericks (10.9.4)

Posted on Jul 28, 2014 6:42 PM

Reply

Answer 1

Linc Davis

Level 10

209,739 points

Jul 28, 2014 7:20 PM in response to dhuber82

Triple-click anywhere in the line below on this page to select it:

/.Spotlight-V100

Right-click or control-click the line and select

Services ▹ Open

from the contextual menu.* A folder should open with an item selected. Move the selected item to the Trash. You'll be prompted for your login password. Restart the computer, empty the Trash, and test.

*If you don't see the contextual menu item, copy the selected text to the Clipboard by pressing the key combination command-C. In the Finder, select

Go ▹ Go to Folder...

from the menu bar and paste into the box that opens by pressing command-V. You won't see what you pasted because a line break is included. Press return.

Reply

Answer 2

dhuber82 Author

Level 1

0 points

Jul 28, 2014 11:12 PM in response to Linc Davis

Thanks Linc! I've followed the instructions and Spotlight is now rebuilding the index after the restart. I'll report back whether or not the problem returns.

Reply

Answer 3

dhuber82 Author

Level 1

0 points

Jul 30, 2014 6:31 PM in response to Linc Davis

Problem reoccurred ... this time after ~2 days. Again, I could not pin it down to anything specific that I was doing at the time. Are there any tasks that I should look out for after Spotlight crashes that might help to diagnose the problem?

I forgot to mention in the original post that I've also attempted to verify/repair disc permissions, which did not fix the problem.

Reply

Answer 4

Linc Davis

Level 10

209,739 points

Jul 30, 2014 6:56 PM in response to dhuber82

Step 1

These instructions must be carried out as an administrator. If you have only one user account, you are the administrator.

Triple-click anywhere in the line below on this page to select it:

syslog -F '$Time $Message' -k Sender mdworker -o -k Message Rne Norm -k Sender mds | tail | pbcopy

Copy the selected text to the Clipboard by pressing the key combination command-C.

Launch the built-in Terminal application in any of the following ways:

☞ Enter the first few letters of its name into a Spotlight search. Select it in the results (it should be at the top.)

☞ In the Finder, select Go ▹ Utilities from the menu bar, or press the key combination shift-command-U. The application is in the folder that opens.

☞ Open LaunchPad. Click Utilities, then Terminal in the icon grid.

Paste into the Terminal window by pressing the key combination command-V. I've tested these instructions only with the Safari web browser. If you use another browser, you may have to press the return key after pasting.

The command may take a noticeable amount of time to run. Wait for a new line ending in a dollar sign ($) to appear.

The output of the command will be automatically copied to the Clipboard. If the command produced no output, the Clipboard will be empty. Paste into a reply to this message.

The Terminal window doesn't show the output. Please don't copy anything from there.

If any personal information appears in the output, anonymize before posting.

Step 2

Enter the following command as in Step 1 and post the output:

mdutil -as 2>&- | pbcopy

You can then quit Terminal.

Step 3

Launch the Console application in the same way you launched Terminal. In the Console window, look under the heading DIAGNOSTIC AND USAGE INFORMATION on the left for crash reports related to Spotlight. If you don't see that heading, select

View ▹ Show Log List

from the menu bar. A Spotlight crash report has a name beginning in "mds" or "mdworker" and ending in ".crash". Select the most recent such report, if any, from the System and User subcategories and post the entire contents—the text, please, not a screenshot. In the interest of privacy, I suggest that, before posting, you edit out the “Anonymous UUID,” a long string of letters, numbers, and dashes in the header of the report, if it’s present (it may not be.)

Please don’t post any other kind of diagnostic report, such as hang logs—they're very long and not helpful.

Reply

Answer 5

dhuber82 Author

Level 1

0 points

Jul 30, 2014 10:45 PM in response to Linc Davis

Step 1:

Jul 30 11:05:13 (Warning) Server: No stores registered for metascope "kMDQueryScopeComputer"

Jul 30 11:05:17 (Warning) Server: No stores registered for metascope "kMDQueryScopeComputer"

Jul 30 11:05:18 (Warning) Server: No stores registered for metascope "kMDQueryScopeComputer"

Jul 30 11:05:19 (Warning) Server: No stores registered for metascope "kMDQueryScopeComputer"

Jul 30 11:05:20 (Warning) Server: No stores registered for metascope "kMDQueryScopeComputerIndexed"

Jul 30 11:05:24 (Warning) Server: No stores registered for metascope "kMDQueryScopeComputer"

Step 2:

mdutil -as 2>&- | pbcopy

Ambiguous output redirect.

Step 3:

no mds or mdworker crash reports are present - I only see reports named:

dbus-daemon,diskarbitrationd,mds,mds_stores_[followed by some more].shutdownStall

mds_stores[followed by some more].spin

I should note that I've done all these steps with indexing turned off (i.e. I restarted the computer after Spotlight crashed). I'll reindex tonight and can repost the same test after Spotlight actually crashes, if that is more helpful.

Reply

Answer 6

Linc Davis

Level 10

209,739 points

Jul 30, 2014 10:51 PM in response to dhuber82

Yes, and please use the default bash shell, not tcsh.

Reply

Answer 7

dhuber82 Author

Level 1

0 points

Aug 1, 2014 4:51 PM in response to Linc Davis

Here are the results right after spotlight stopped working, using the bash shell:

Step 1:

Jul 31 03:41:39 FontImporter: Validation Result - "(

kATSFontTestSeverityInformation,

kATSFontTestSeverityFatalError,

kATSFontTestSeverityMinorError,

kATSFontTestSeverityTechnicalError

)".

Jul 31 05:53:55 FontImporter: Validation failed - "/sw/share/texmf-dist/doc/fonts/cm-unicode/Fontmap.CMU.otf".

Jul 31 05:53:55 FontImporter: Validation Result - "(

kATSFontTestSeverityFatalError

)".

Step 2:

/:

Indexing enabled.

/.MobileBackups:

No index.

/Volumes/MobileBackups:

Index is read-only.

/Volumes/MobileBackups/Backups.backupdb:

Index is read-only.

Step 3:

As before, I don't see .crash reports related to mds or mdworker.

Reply

Answer 8

Linc Davis

Level 10

209,739 points

Aug 1, 2014 6:01 PM in response to dhuber82

Back up all data, then delete the file below and see whether there's an improvement:

/sw/share/texmf-dist/doc/fonts/cm-unicode/Fontmap.CMU.otf

Reply

Answer 9

dhuber82 Author

Level 1

0 points

Aug 3, 2014 11:58 AM in response to Linc Davis

I removed the file, re-indexed, and Spotlight crashed again (within the usual ~1 day cycle). Here is the log from Step 1 I received this time:

Aug 2 00:39:39 zip importer encountered an error (2) scanning "/Users/daniel/Library/Mail/V2/IMAP-danxhuber@imap.gmail.com/[Gmail].mbox/Sent Mail.mbox/[anonymized]/Data/3/1/Attachments/13187/2.6/figures.zip".

Aug 2 00:39:39 zip importer encountered an error (2) scanning "/Users/daniel/Library/Mail/V2/IMAP-danxhuber@imap.gmail.com/[Gmail].mbox/Sent Mail.mbox/[anonymized]/Data/2/1/Attachments/12720/2/janeen.zip".

Aug 2 00:39:39 zip importer encountered an error (2) scanning "/Users/daniel/Library/Mail/V2/IMAP-danxhuber@imap.gmail.com/[Gmail].mbox/Sent Mail.mbox/[anonymized]/Data/2/1/Attachments/12720/3/cheryl.zip".

Aug 2 04:53:39 FontImporter: Validation failed - "/Users/daniel/science/MOST/old/qfits/qfits-5.3.0/latex/Helvetica.ttf".

Aug 2 04:53:39 FontImporter: Validation Result - "(

kATSFontTestSeverityInformation,

kATSFontTestSeverityFatalError,

kATSFontTestSeverityMinorError,

kATSFontTestSeverityTechnicalError

)".

None of these files are anything I have used/moved/renamed/etc recently. Step 2 & 3 yielded the same results as before. I've also inspected all messages in the Console right after the crash occurred, but couldn't find tasks related to mds or mdworker.

I noticed that Spotlight is actually not completely crashed - it does seem to find everything in the Application folder (Safari, etc) after the crash occurred. However, none of the files in my home directory show up, and searches in e.g. Mail don't return any results.

Reply

Answer 10

Linc Davis

Level 10

209,739 points

Aug 3, 2014 12:12 PM in response to dhuber82

The Spotlight index can be corrupted when bad data is encountered. It shouldn't happen, but it does. The last log extract shows several bad attachments in the Mail database, and another bad font in your home folder. Until you get rid of all that, I would expect the problem to keep recurring.

Reply

Answer 11

dhuber82 Author

Level 1

0 points

Aug 7, 2014 10:30 AM in response to Linc Davis

So far so good - after removing the last batch of bad files Spotlight has been running fine for the past 3 days. I'll give it another week or so, and if it is still unaffected I think we can call this problem solved.

Reply

Answer 12

dhuber82 Author

Level 1

0 points

Sep 12, 2014 12:37 AM in response to dhuber82

Looks like I was little too optimistic. A few days after the last post, Spotlight crashed again with the log pointing to more corrupt files in my Gmail attachments. Reading up on some other posts reporting problems with Mac Mail corrupting attachments from Gmail, I removed both the Gmail inbox and sent folder from the Spotlight search index. This worked for about a month, until a few days ago Spotlight again started crashing on a nearly daily basis. This time, the logs show no entries related to corrupt files (neither in Gmail nor anywhere else), and the console again does not report an actual crash of mds.

I've pasted the 3 most recent logs after Spotlight crashes below. It seems that we're back to square one with finding the root of this problem ... any help would be greatly appreciated.

Sep 8 12:18:41 (Warning) Server: No stores registered for metascope "kMDQueryScopeComputer"

Sep 8 12:18:43 (Warning) Server: No stores registered for metascope "kMDQueryScopeComputer"

Sep 8 12:18:44 (Warning) Server: No stores registered for metascope "kMDQueryScopeComputer"

Sep 8 12:18:45 (Warning) Server: No stores registered for metascope "kMDQueryScopeComputerIndexed"

Sep 8 12:19:56 ImageIO: _CGImagePluginInitGIF malformed GIF file (390 x 173) - [canvasSize: 67470 fileSize: 73 ratio: 924]

Sep 10 17:45:45 Cannot create fileport: -1: Unknown error: -1

Sep 10 17:45:45 (Error) Server: *** textSnippetFDs Failed to create FD! 0 - oid:7710163112081293313 - fd:0

Sep 10 17:45:45 Cannot create fileport: -1: Unknown error: -1

Sep 10 17:45:45 (Error) Server: *** textSnippetFDs Failed to create FD! 0 - oid:7710163112081293313 - fd:0

Sep 10 17:45:45 Cannot create fileport: -1: Unknown error: -1

Sep 10 17:45:45 (Error) Server: *** textSnippetFDs Failed to create FD! 0 - oid:7710163112081293313 - fd:0

Sep 10 17:45:45 Cannot create fileport: -1: Unknown error: -1

Sep 10 17:45:45 (Error) Server: *** textSnippetFDs Failed to create FD! 0 - oid:7710163112081293313 - fd:0

Sep 10 17:46:04 Cannot create fileport: -1: Unknown error: -1

Sep 10 17:46:04 (Error) Server: *** textSnippetFDs Failed to create FD! 0 - oid:7710163112081293313 - fd:32686

Sep 11 11:57:24 (Error) Server: ==== XPC handleXPCMessage XPC_ERROR_CONNECTION_INVALID

Sep 11 11:58:37 (Warning) Server: No stores registered for metascope "kMDQueryScopeComputer"

Reply

Answer 13

dhuber82 Author

Level 1

0 points

Apr 23, 2015 7:04 PM in response to dhuber82

9 months on from my original post, I'm sad to report that this problem still exists (now running the latest version of Yosemite). The longest stretch I've gone with working Spotlight was probably about a month, and lately I've been back to something like a few days. I have re-indexed dozens of times, repaired disk permissions, prayed that software updates would address this, but to no avail. Here are a few specific things I have tried:

1) Removing all non-essential directories from the index (in the privacy section of the spotlight preference pane) and re-indexing.

2) Tracking the console log during each re-index overnight, removing all files that would throw up an error for mds/mdworker such as fonts (which also seems to be a long standing problem: http://www.dextechs.com/osx105spotlight.html), and re-indexing over again.

3) Relaunching spotlight using:

sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

once it crashed. This works temporarily, but typically not longer than when re-indexing from scratch.

As mentioned before, there are no mds or mdworker processes/messages in the system log that seem to be related to what is causing spotlight to actually crash. For example, here are the mds and mdworker messages from this morning (from the console):

4/24/15 8:00:11.604 AM mdworker[11943]: (ImportBailout.Error:1325) Asked to exit for Diskarb

4/24/15 8:00:11.604 AM mdworker[11941]: (ImportBailout.Error:1325) Asked to exit for Diskarb

4/24/15 8:00:11.608 AM mdworker[13403]: (ImportBailout.Error:1325) Asked to exit for Diskarb

4/24/15 8:00:11.612 AM mdworker[12539]: (ImportBailout.Error:1325) Asked to exit for Diskarb

4/24/15 8:00:11.618 AM mdworker[13438]: (ImportBailout.Error:1325) Asked to exit for Diskarb

4/24/15 8:00:11.622 AM mdworker[11940]: (ImportBailout.Error:1325) Asked to exit for Diskarb

4/24/15 10:14:39.359 AM mdworker[13549]: (ImportBailout.Error:1325) Asked to exit for Diskarb

4/24/15 10:14:39.363 AM mdworker[13557]: (ImportBailout.Error:1325) Asked to exit for Diskarb

4/24/15 10:14:39.364 AM mdworker[13570]: (ImportBailout.Error:1325) Asked to exit for Diskarb

4/24/15 10:14:39.368 AM mdworker[13569]: (ImportBailout.Error:1325) Asked to exit for Diskarb

4/24/15 10:14:39.372 AM mdworker[13934]: (ImportBailout.Error:1325) Asked to exit for Diskarb

4/24/15 10:14:39.373 AM mdworker[13840]: (ImportBailout.Error:1325) Asked to exit for Diskarb

4/24/15 10:14:39.375 AM mdworker32[13998]: (ImportBailout.Error:1325) Asked to exit for Diskarb

4/24/15 10:14:39.392 AM mdworker[14031]: (ImportBailout.Error:1325) Asked to exit for Diskarb

4/24/15 7:33:50.583 AM SubmitDiagInfo[495]: Removed expired problem report: file:///Library/Logs/DiagnosticReports/mds_2015-03-25-005325_Daniels-MacBook-Pr o-2.crash

4/24/15 7:33:50.587 AM SubmitDiagInfo[495]: Removed expired problem report: file:///Library/Logs/DiagnosticReports/mds_stores_2015-03-25-062325_Daniels-Mac Book-Pro-2.crash

4/24/15 8:00:11.596 AM mds[58]: (Volume.Normal:2464) volume:0x7fd2ab81c000 ********** Bootstrapped Creating a default store:0 SpotLoc:(null) SpotVerLoc:(null) occlude:0 /Volumes/Recovery HD

4/24/15 10:14:39.231 AM mds[58]: (Volume.Normal:2464) volume:0x7fd2ab803000 ********** Bootstrapped Creating a default store:0 SpotLoc:(null) SpotVerLoc:(null) occlude:0 /Volumes/Recovery HD

Spotlight worked fine before 10am and then crashed, but none of the above seems to be related to why this actually happened. There are a few mds_stores crash reports (such as the one mentioned above), but none of them coincide with when Spotlight stops working. Are there other processes except for mds and mdworker that control spotlight?

Any help/advice would be very much appreciated - I am at this point at a complete loss on how to fix this. It's incredibly depressing that something as fundamental as a search tool does not properly work in an operating system.

Reply