Spotlight server not working properly

We have all Leopard clients so we haven't seen the Tiger issue you've described.

I noticed the spotlight ACL, too. I suspect that's part of the solution.

mm

It appears our issue has something to do with mdworker using the fontimporter process to try and validate a font, and then that validation fails. This can be seen in Console>Log Database Queries>All Messages.

Log Files>/Library/Logs>CrashReporter shows mdworker crashing during that time.

We have lots of fonts on our shares. I wasn't aware that Leopard server also tries to validate fonts as Leopard client does. I wonder if some of these fonts are not Leopard-worthy and have issues, or if it's just fonts in general that cause Spotlight to have issues when trying to index them.

I noticed the Spotlight ACL that was being added to our new shares as well with OS X Server 10.5.6. The Spotlight user (local UID=89) has been around for some time, since at least OS X Server 10.5.3, but this new ACL addition seems to have appeared in OS X Server 10.5.6, possibly related to changes in how content is indexed.

I was able to clear up our long-standing Spotlight problems for Leopard clients by making sure the Spotlight user has read privileges on all files needing indexing, and list+traverse privileges on all folders that contain them. I did this by adding explicit ACLs on all network shares. Although Tiger-based clients are still getting zero Spotlight matches from network shares, I'm hoping that will clear up with a reboot of our servers as it did for district6.

A few comments about Apple's Spotlight ACL are in order:

First of all, the ACL doesn't appear on its own when you upgrade to OS X Server 10.5.6. I found that I had to turn sharing off, then on, for each network share for the new ACL to appear.

Second, the ACL only applies to the topmost folder for shares with existing content. Inheritance is enabled in the ACL which will apply to new folders that are created from the top level, but this won't apply other folders in an existing hierarchy. You either need to propagate permissions yourself, or manually execute a shell command as shown below.

Third, the ACL only grants Spotlight directory list+traverse privileges to enclosing folders (aka "r-x" to directories in POSIX terms), but does NOT explicitly grant read privileges for Spotlight to access files. In our case, we have POSIX world read privileges turned off on all files and folders on most network shares, and use ACLs which require users to be members of our groups for any access. Although you can add the Spotlight user to these groups to grant access, we still ended up with a couple of spots on our shares where Spotlight didn't have access (such as in network-based Drop folders where individual users, and not groups, had read access). The best solution for us was to propagate a Spotlight ACL to grant Spotlight specific rights. You can either add this at the top of each network share (or volume) and propagate it in Server Admin, or as we did with a little shell programming (note that this command will fail for any folders containing existing ACLs that are not in canonical order to begin with):

sudo chmod -R +ai "user:_spotlight allow read,readattr,list,search,file inherit,directoryinherit" /path/to/share

or if you prefer for the Server Admin GUI:

[ ] Administration

[-] Read
[X] Read Attributes
[ ] Read Extended Attributes
[X] List Folder Contents (Read Data)
[X] Traverse Folder (Execute File)
[ ] Read Permissions

[ ] Write

[X] Inheritance
[X] Apply to this folder
[X] Apply to child folders
[X] Apply to child files
[X] Apply to all descendants

Without paying attention to the Spotlight user for indexing, what you end up with is that only the files that are owned by LOCAL server users (such as the Administrator) will be included in the Spotlight index. All content for network clients, and those with network or mobile home folders will be excluded. This explains what we orginally saw: our Spotlight indices were way too small and only included files owned by the administrator and certain other server-based local users. In fact, there is an mdworker process that is launched for each local user, and a separate one for the Spotlight user which catalogs all the other files, such as those owned by users running from client machines or who own network-based or mobile home folders. Their content will be exluded from the Spotlight catalog unless the Spotlight user can read listtraverse all the necessary content.

One way to do this as others have pointed out is to grant world read privileges to all files either in POSIX or via ACLs. Though this is a simple fix, this may open up security holes that you don't want. Another way, which we did at first, was to add the Spotlight user to your security groups, but as I mentioned this didn't work for network Drop Folders and other spots where special ACLs were in effect which didn't grant group read listtraverse privileges. The best solution for us was to propagate the type of ACL previously shown.

Hope this helps solve any problems you're having with Spotlight.

The "mdls" and "mdfind" tools, when executed from the OS X Server side, are a great way to quickly determine whether or not Spotlight Server is working properly.

Steviestar (and others) have pointed out that "mdls" returns rather sparse file-related information if a file is excluded in the Spotlight index (I count exactly 15 attributes returned using OS X Server 10.5.x), and fatter information is returned if the content for the file is included in the Spotlight index. What I find helpful is to simply ask for the value of the kMDItemContentType attribute, as shown below. This seems to be a simple way to quickly determine if the content is in the index: mdls shows "(null)" if the content hasn't been indexed, and the content type if it has been indexed:

mdls -name kMDItemContentType /path/to/file

I don't trust mdls when run from the client side as there's additional layers of software and networking going on, but of course this is the definitive way to know if Spotlight is running properly at both ends.

The "mdfind" command is also a great way to figure out if your Spotlight Server index includes everything you want. For example, you can use the following command to locate all PDFs included in the index, allowing you to spot major indexing holes relatively quickly:

mdfind "kMDItemContentType = com.adobe.pdf"

Hope this helps.

hopefully I've read all the posts, and I'm not repeating anything here, but...

THIS IS APPARENTLY A BUG IN 10.5.6 Software Update. Clients on our network were having no trouble searching until they updated to 10.5.6... One machine is still at 10.5.3 and it searches just fine.

Good luck.

Message was edited by: tiiger

Yup same here. Leopard server with tiger clients. Suddenly tiger clients begin to return 0 results until Leopard server is rebooted...

Im frustrated.

BTW, Where in the console can I find the logs for MDS?

same problem with a fresh new 10.5.6 xserver. Spotlight is running well with 10.5 client but it's very very very long on 10.4 clients...

applied server 10.5.6 it seems to work on the share point but searching on some specific folders under the share does not find the anything? has anybody experience this?

how are you guys searching from the root share or you can also search specific folders under the root share?

Server = 10.5.6
clients = 10.5.6, plus one 10.5.5

Hey guys,

The spotlight server, in my case does not search on my 10.5.6, searches only on mounted volumes. When I click in finder on my server, there are many shares, If I only connect to one the search of spotlight on shared, searches only on mounted volumes!!!. If I connect to all volumes under the "_my server_" in finder, the search will look under SHARED in all *network volumes mounted*.

Dunno If I make any sense to ya all here ... but for me it solves the problem of searching under Share in finder. Also did +sudo mdutil -s /Volumes/*+ and a couple of other commands to ensure that spotlight server is indexing the volumes on witch I search thing on server.

If anyone did not understood this reply please feel free to comment.

Message was edited by: Georgemaxim