Spotlight forever indexing

Please read this whole message before doing anything.

This procedure is a test, not a solution. Don’t be disappointed when you find that nothing has changed after you complete it.

Step 1

The purpose of this step is to determine whether the problem is localized to your user account.

Enable guest logins*, log out of your account, and log in as Guest. For instructions, launch the System Preferences application, select Help from the menu bar, and enter “Set up guest users” (without the quotes) in the search box. Don't use the Safari-only “Guest User” login created by “Find My Mac.”

While logged in as Guest, you won’t have access to any of your personal files or settings. Applications will behave as if you were running them for the first time. Don’t be alarmed by this; it’s normal. If you need any passwords or other personal data in order to complete the test, memorize, print, or write them down before you begin.

Test while logged in as Guest. Same problem(s)?

After testing, log out of the guest account and, in your own account, disable it if you wish. Any files you created in the guest account will be deleted automatically when you log out of it.

*Note: If you’ve activated “Find My Mac” or FileVault, then you can’t enable the Guest account. The “Guest User” login created by “Find My Mac” is not the same. Create a new account in which to test, and delete it, including its home folder, after testing.

Step 2

The purpose of this step is to determine whether the problem is caused by third-party system modifications that load automatically at startup or login.

Disconnect all wired peripherals except those needed for the test, and remove all aftermarket expansion cards. Boot in safe mode* and log in to the account with the problem. The instructions provided by Apple are as follows:

1. Be sure your Mac is shut down.
2. Press the power button.
3. Immediately after you hear the startup tone, hold the Shift key. The Shift key should be held as soon as possible after the startup tone, but not before the tone.
4. Release the Shift key when you see the gray Apple icon and the progress indicator (looks like a spinning gear).

Safe mode is much slower to boot and run than normal, and some things won’t work at all, including wireless networking on certain Macs.

The login screen appears even if you usually log in automatically. You must know your login password in order to log in. If you’ve forgotten the password, you will need to reset it before you begin.

*Note: If FileVault is enabled, or if a firmware password is set, you can’t boot in safe mode.

Test while in safe mode. Same problem(s)?

After testing, reboot as usual (i.e., not in safe mode) and verify that you still have the problem. Post the results of steps 1 and 2.

Remove the rest of the useless "Tech Tool" crapware according to the developer's instructions. If the problem doesn't return, it's solved.

I don't understand how software which does not conflict with spotlight, prior to 10.7.5, is crapware?

I mean, one day the system is functioning fine and then Software Update advises us to update to 10.7.5, then Spotlight goes nuts, and you say at that point all the software on our systems conflicting with spotlight is "crapware". Why does this make no sense?

I'm having this problem since 10.7.5 as well. ~/Library/Spotlight does not exist (only one user account), and /Library/Spotlight contains no importers. So it could be a 3rd party application importer: Colloquy, Evernote, LibreOffice?

Console reports this:

Oct 1 14:51:55 mingisapsycho mdworker[317]: Unable to talk to lsboxd

Oct 1 14:52:09 mingisapsycho mds[66]: (/.Spotlight-V100/Store-V2/92954810-8470-4933-A363-3E87B3D6F841)(Error) IndexStore in bool setLocalizedAttributesLocked(datastore_info*, db_obj**, const __CFString**, const __CFDictionary**, CFIndex, StoreOptions, const __CFDictionary*, bool, bool, const __CFArray*):Error:7 setting field:com_apple_system_prefs_keywords for oid:292103

and

Oct 1 14:53:49 mingisapsycho mds[66]: (Error) ImportServer: <MDSImporterWrangler: 0x7fc6a161b120> still waiting for plug-in info from mdworker

Oct 1 14:54:54 mingisapsycho mds[66]: (Error) ImportServer: <MDSImporterWrangler: 0x7fc6a143a800> still waiting for plug-in info from mdworker

At the same time as the waiting for plug-in, 'sudo opensnoop -n mdworker' reports this:

501 364 mdworker 3 /dev/autofs_nowait

501 364 mdworker 4 /Users/chris/.CFUserTextEncoding

501 364 mdworker 3 /dev/autofs_nowait

501 364 mdworker 4 /Users/chris/.CFUserTextEncoding

501 364 mdworker -1 /Library/Sandbox/Profiles/mdworker.sb

501 364 mdworker -1 /System/Library/Sandbox/Profiles/mdworker.sb

501 364 mdworker 4 /usr/share/sandbox/mdworker.sb

501 364 mdworker 5 /var/folders/9h/nxvdj4k96zsg_6pkghc465lc0000gn/C//sandbox-cache.db

About 10 minutes later in console:

Oct 1 15:04:42 mingisapsycho mdworker[379]: Unable to talk to lsboxd

And at the same time with opensnoop:

501 380 mdworker -1 /Library/Sandbox/Profiles/mdworker.sb

501 380 mdworker -1 /System/Library/Sandbox/Profiles/mdworker.sb

501 380 mdworker 4 /usr/share/sandbox/mdworker.sb

501 380 mdworker 5 /var/folders/9h/nxvdj4k96zsg_6pkghc465lc0000gn/C//sandbox-cache.db

89 379 mdworker 8 /var/folders/zz/zyxvpxvq6csfxvn_n00000b400002s/C//mds/mds.lock

89 379 mdworker 12 /var/folders/zz/zyxvpxvq6csfxvn_n00000b400002s/C//mds/mdsObject.db_

89 379 mdworker 13 /private/var/db/mds/system/mdsObject.db

89 379 mdworker 12 /var/folders/zz/zyxvpxvq6csfxvn_n00000b400002s/C//mds/mdsDirectory.db_

89 379 mdworker 13 /private/var/db/mds/system/mdsDirectory.db

89 379 mdworker 12 /var/folders/zz/zyxvpxvq6csfxvn_n00000b400002s/C//mds/mdsObject.db

89 379 mdworker 12 /System/Library/Security

89 379 mdworker -1 /var/empty/Library/Security

89 379 mdworker 8 /var/folders/zz/zyxvpxvq6csfxvn_n00000b400002s/C//mds/mdsDirectory.db

501 380 mdworker 4 /System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework /Versions/A/Support

501 380 mdworker 4 /System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework /Versions/A/Support/mdworker

It looks like mdworker processes get killed after hanging and mds starts a new instance. All instances are hanging at the same point, the sandbox-cache.db. The bits before that differ very little. I kinda feel like nuking that .GlobalPreferences.plist file as well as the user equivalent and the user ByHost folder, rebooting and see if that fixes it. If not, then yank the three 3rd party apps with mdimporters: Evernote, Colloquy, and LibreOffice. For some reason mdworkers are being sandboxed, is what I think it comes down to. I just don't know why.

89 430 mdworker -1 /etc/sysinfo.conf

89 430 mdworker 4 /System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework /Versions/A/Support

89 430 mdworker 4 /System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework /Versions/A/Support/mdworker

89 430 mdworker 3 /dev/autofs_nowait

89 430 mdworker -1 /var/empty/.CFUserTextEncoding

89 430 mdworker 3 /dev/autofs_nowait

89 430 mdworker -1 /var/empty/.CFUserTextEncoding

89 430 mdworker 3 /System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework /Versions/A/Support/libmdworker.dylib

89 430 mdworker 3 /dev/dtracehelper

89 430 mdworker 4 /.vol/234881028/228459/mdworker/..namedfork/rsrc

89 430 mdworker 4 /.vol/234881028/228459/mdworker/..namedfork/rsrc

89 430 mdworker 4 /.vol/234881028/228459/mdworker/..namedfork/rsrc

89 430 mdworker 4 /Library/Preferences/.GlobalPreferences.plist

89 430 mdworker -1 /Library/Sandbox/Profiles/mdworker.sb

89 430 mdworker -1 /System/Library/Sandbox/Profiles/mdworker.sb

89 430 mdworker 4 /usr/share/sandbox/mdworker.sb

89 430 mdworker 5 /var/folders/zz/zyxvpxvq6csfxvn_n00000b400002s/C//sandbox-cache.db

The time estimate for completion is 5 weeks. In the meantime spotlight is useless.

a. Deleted user ByHost, .GlobalPreferences, and system .GlobalPreferences. Date/Time information in the menubar is now screwed up "it says 2,2012 10 01" for the time. Clicking on Date & Time panel in System Preferences causes System Preferences to crash.

b. Removed Evernote, Colloquy, LibreOffice to the trash.

c. sudo mdutil -E / to remove the index thus far

d. Confirmed that opensnoop is not finding their .mdimporter plugins.

e. No change in Spotlight lengthy re-indexing or opensnoop messages

501 343 mdworker -1 /Library/Sandbox/Profiles/mdworker.sb

501 343 mdworker -1 /System/Library/Sandbox/Profiles/mdworker.sb

501 343 mdworker 4 /System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework /Versions/A/Support

501 343 mdworker 4 /System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework /Versions/A/Support/mdworker

501 343 mdworker 3 /System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework /Versions/A/Support/libmdworker.dylib

501 343 mdworker 3 /dev/dtracehelper

501 343 mdworker -1 /etc/sysinfo.conf

501 343 mdworker 3 /dev/autofs_nowait

501 343 mdworker 4 /Users/chris/.CFUserTextEncoding

501 343 mdworker 3 /dev/autofs_nowait

501 343 mdworker 4 /Users/chris/.CFUserTextEncoding

501 343 mdworker 4 /.vol/234881028/228459/mdworker/..namedfork/rsrc

501 343 mdworker 4 /.vol/234881028/228459/mdworker/..namedfork/rsrc

501 343 mdworker 4 /.vol/234881028/228459/mdworker/..namedfork/rsrc

501 343 mdworker 4 /usr/share/sandbox/mdworker.sb

501 343 mdworker 5 /var/folders/9h/nxvdj4k96zsg_6pkghc465lc0000gn/C//sandbox-cache.db

501 343 mdworker 4 /Users/chris/Library/Preferences/ByHost/.GlobalPreferences.A63BCE32-A196-50B8-A 7C8-31862B971594.plist

501 343 mdworker 4 /Users/chris/Library/Preferences/.GlobalPreferences.plist

501 343 mdworker 4 /Library/Preferences/.GlobalPreferences.plist

So the cause is not any of the things I deleted, nor the 3rd party importers from Evernote, Colloquy, or LibreOffice.

kilgo wrote:

I don't understand how software which does not conflict with spotlight, prior to 10.7.5, is crapware?

I mean, one day the system is functioning fine and then Software Update advises us to update to 10.7.5, then Spotlight goes nuts, and you say at that point all the software on our systems conflicting with spotlight is "crapware". Why does this make no sense?

The obvious answer to that is that the 'Crapware' has not been updated from Lion to Mountain Lion yet

Christopher Murphy wrote:

The time estimate for completion is 5 weeks. In the meantime spotlight is useless.

"Useless" is actually not true. At a certain point, there is enough indexed that some searches do work. Mine has "19 hours remaining" and there's quite a bit that I can find now, though not everything.

I am not sure what to make of the info in the later posts of this thread. I don't have the third-party mdimporters listed here, and in my case, the one confirmed cause of indexing stopping is when a volume is mounted. Any volume: A disk, a disk image, even a disk image mounted by an updater.

When any volume mounts, indexing stops. I check Spotlight status with the Terminal and it says indexing is disabled on my boot volume. I run the terminal command to restart indexing, and it starts from scratch ("x days") with no search results available. If any volume is mounted again, indexing will be disabled on my boot volume only and I have to start it in Terminal again.

And none of this happened before 10.7.5.

I might have a working "fix". Based on the constant stalling on sandbox-cache.db I decided to take a chance and delete all folders found in /var/folders using the command. What I actually did, in order:

Restored Evernote, Colloquy, LibreOffice back to /Applications (from the Trash).

Quit all applications except Terminal, issued the following two commands:

sudo mdutil -i off /

sudo mdutil -E /

sudo rm -Rf /var/folders/*

Reboot.

sudo mdutil -i on /

There were no further stalls on sandbox-cache.db according to opensnoop, and no further lsboxd complaints in Console. mds and mdworker processes have been very busy, typically 40-100% CPU, unlike before. And now the index estimate is variably saying 6 to 19 minutes for completion instead of 3 days to 5 weeks. So hopefully in an hour or so (the typical amount of time for my system to do a Spotlight index rebuild) I will know if this has really worked.

Now, blowing away the folders in /var/folders may have other consequences, possibly negative, for sandboxing. But the files therein appear to be recreated from scratch after reboot to no ill effect or complaints in Console. But let's just say, the suffering bleeding edgers may end up bleeding a lot more by trying this. Any volunteers?

Also posted this response in : Apple Support Communities > Mac OS & System Software > OS X Lion > Re: Spotlight not working correctly

I'm going to post updates to this thread instead of double posting.