Announcement: Upgrade to macOS Mojave

With features like Dark Mode, Stacks, and four new built-in apps, macOS Mojave helps you get more out of every click. 
Find out how to upgrade to macOS Mojave > https://support.apple.com/macos/mojave

Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

Question:

Question: How can I make OSX index a network share for spotlight?

How can I make OSX index a network share for spotlight?


I am using a QNAP NAS with SMB network shares, which I mount with OS X Mavericks. They are not searchable, as far as I know because of the indexing of network share is by default not activated.


To force the OS to index them I activated this via mdutils. But this is no solutions. This makes spotlight to index the share once. But I want spotlight to index it always and keep it up to date. I have a huge amount of documents & fotographs, which is very difficult to manage without being searchable.


Unfortunately the only solution I found in the www was the mdutils-way, which seems not to work, maybe only works not for me.



I would appreciate any assistance,




Thank you,


Stefan

iMac (27-inch Mid 2011), OS X Mavericks (10.9.1)

Posted on

Reply
Question marked as Apple Recommended
Answer:
Answer:

The way Spotlight works on a Mac server is that -


  1. There is a special server process running to do the indexing automatically - normally by monitoring file system changes i.e. modifying a file or creating/deleting files
  2. The use of a special Spotlight system account and the automatic addition via ACL records to allow this system account to have access to every file and folder


With a server obviously the usual situation is that there are multiple different users and each user normally has different access permissions such that userA may not be able to access files owned by userB. Therefore userA is not going to be able to access and index files restricted to userB, hence as described above the special Spotlight system account on a Mac server.


Spotlight on a Mac server is also clever enough to not only be able to index files owned by everyone, but to also only return the search results that user is allowed to access, therefore you will not see in the list of results files private to other users.


Realistically this sort of system has to be done on the server itself, therefore it is the server supplier who has to provide this capability. So in your case QNAP.


Unfortunately Spotlight is proprietary to Apple so no-one else can add it to their product. You would still be able to do a basic filename search but not a file content search. There are some third-party products you can buy which can do their own file content indexing and these provide their own search clients. I used to work for a company and run the IT for them and we used a tool called 'Sonar' which did this. See http://www.virginiasystems.com/products_s.html (I would not advise buying this particular solution now - it is possitively ancient and Spotlight would be the better choice.)


A possible option would be to have a Mac server and have the Mac server mount the QNAP as a drive, and then have the Mac server re-share the QNAP. Mac clients would then access it via the Mac server. The Mac server would then handle all the user-authentication i.e. logins, and would also do the Spotlight indexing. This would add a network traffic overhead and this could be reduced by having two network connections on the Mac server, one to the LAN with the Mac clients, and a second dedicated on just to the QNAP.


Some people use iSCSI or NFS to connect to the QNAP (or similar NAS system), while the Mac clients would use AFP or SMB to connect to the Mac server as normal.

Posted on

Page content loaded

Jun 11, 2014 9:55 AM in response to Bhaskar Narula In response to Bhaskar Narula

Sorry, but what do you mean by "why sould it be acceptable"?


No, I found no solution but some catalogueing-tool named "neofinder".

By this I am able to index my shares, or even external drives, and search them. for external drives thats a acceptable solution, because it gives me a way to search drives without plugin them in.


but network shares should be searchable online (instantly) & offline.


from my point of view spotlight should be able to handle these two cases. but I seem to be the only one who has a problem with this.

Jun 11, 2014 9:55 AM

Reply Helpful
Question marked as Apple Recommended

Jun 13, 2014 2:39 AM in response to stefanfromeuskirchen In response to stefanfromeuskirchen

The way Spotlight works on a Mac server is that -


  1. There is a special server process running to do the indexing automatically - normally by monitoring file system changes i.e. modifying a file or creating/deleting files
  2. The use of a special Spotlight system account and the automatic addition via ACL records to allow this system account to have access to every file and folder


With a server obviously the usual situation is that there are multiple different users and each user normally has different access permissions such that userA may not be able to access files owned by userB. Therefore userA is not going to be able to access and index files restricted to userB, hence as described above the special Spotlight system account on a Mac server.


Spotlight on a Mac server is also clever enough to not only be able to index files owned by everyone, but to also only return the search results that user is allowed to access, therefore you will not see in the list of results files private to other users.


Realistically this sort of system has to be done on the server itself, therefore it is the server supplier who has to provide this capability. So in your case QNAP.


Unfortunately Spotlight is proprietary to Apple so no-one else can add it to their product. You would still be able to do a basic filename search but not a file content search. There are some third-party products you can buy which can do their own file content indexing and these provide their own search clients. I used to work for a company and run the IT for them and we used a tool called 'Sonar' which did this. See http://www.virginiasystems.com/products_s.html (I would not advise buying this particular solution now - it is possitively ancient and Spotlight would be the better choice.)


A possible option would be to have a Mac server and have the Mac server mount the QNAP as a drive, and then have the Mac server re-share the QNAP. Mac clients would then access it via the Mac server. The Mac server would then handle all the user-authentication i.e. logins, and would also do the Spotlight indexing. This would add a network traffic overhead and this could be reduced by having two network connections on the Mac server, one to the LAN with the Mac clients, and a second dedicated on just to the QNAP.


Some people use iSCSI or NFS to connect to the QNAP (or similar NAS system), while the Mac clients would use AFP or SMB to connect to the Mac server as normal.

Jun 13, 2014 2:39 AM

Reply Helpful (2)

Jun 13, 2014 12:20 PM in response to Bhaskar Narula In response to Bhaskar Narula

Bhaskar Narula wrote:


Is there a way to make SL work with a windows server?


If you use a standard Windows server then it does not have built-in Spotight capabilities so again a workaround would be to use a Mac server to 're-share' it and do the indexing. This would be particularly pointless in this case.


There is another option which I was not aware of until I checked just now. I had been aware for many years of a package called ExtremeZ-IP from Group Logic, in fact many years ago I worked for the UK distributor for this product. This product runs on Windows servers and adds AppleShare i.e. AFP capability to Windows Servers. Windows Servers themselves dropped support for AFP many years ago.


It turns out that ExtremeZ-IP does now have the ability to do Spotlight indexing! So a Windows server running ExtremeZ-IP would be a solution.


See http://www.grouplogic.com/resource-center/pdfs/Network-Spotlight-Best-Practices. pdf

and http://www.grouplogic.com/enterprise-file-sharing/mac-windows-file-sharing/


Note: ExtremeZ-IP apparently utilises Windows Search to do the indexing, Windows clients connecting to the same Windows Server via their normal SMB connection can therefore do searching pretty much like Spotlight.


So Windows Server + Windows Search = searching for Windows only

Windows Server + Windows Search + ExtremeZ-IP = searching for Windows AND Mac


PS. ExtremeZ-IP talk about providing access to NAS servers, this is actually exactly the same trick I described to use a Mac to 're-share' the NAS so while it is useful it is not a major feature of ExtremeZ-IP except they use a Windows server to do it.


PPS. Apparently Windows Cluster does not support Windows Search

Jun 13, 2014 12:20 PM

Reply Helpful

Jul 5, 2014 9:42 PM in response to stefanfromeuskirchen In response to stefanfromeuskirchen

My thanks to everyone who has contributed to solving this problem here.


It appears that there still is NOT a good, reliable solution. Please correct me if I'm wrong.


I did find this on http://jonathansblog.co.uk/how-to-enable-spotlight-indexing-on-a-network-drive:

===================================

To enable spotlight indexing on a network drive:

# mdutil /Volumes/name -i on


To disable the indexing of a connected network drive:

# mdutil /Volumes/name -i off


To check the status of indexing on a connected network drive:

# mdutil /Volumes/name -s

===================================

And, I found this on the Synology NAS forums:

---------

cd /Volumes

mdutil shareName -i on

mdimport -Vp ./shareName

---------

where shareName = name of your share

I have used the mdutil /Volumes/name -i on, and it seems to work.

But, if I understand it correctly, it is a one-time indexing -- the index is NOT keep up-to-date with changes.

So, how can we keep the index up-to-date?

My thought was to schedule a Terminal file to execute each night with the mdutil code.

Would this work? If so, how would you setup a scheduled run?

Would it have to be done on each Mac that accesses the Share?

Jul 5, 2014 9:42 PM

Reply Helpful (2)

Oct 31, 2014 1:10 PM in response to stefanfromeuskirchen In response to stefanfromeuskirchen

I am curious if anyone else has experienced the issues that my company is experiencing. We have a mixed client environment, some PCs, and some Macs. Most of the Macs we have are running Mountain Lion, but we have a few newer machines running Mavericks. We have two different main file servers, one that is a Apple Server and a couple of Windows 2008 R2 Servers. Most of the files our designers use are shared out via one of the Windows 2008 R2 servers. Now for the problem. With our designers running ML we were able to Index the windows shares so that spotlight and finder search return quick results of our 100,000+ file shares. However no matter what I do with the Mavericks machines as far as indexing I am unable to get the finder search and spotlight to return any results. I have disabled the index and then the finder search will return results after 5-10 minutes. To top this all off I am currently testing all of our applications with Yosemite and the indexing works like a charm. Unfortunately any of the third party apps such as Easyfind and Pathfinder do not integrate into other apps like Spotlight does for InDesign, etc. Any suggestions or thoughts would be greatly appreciated. One last note, I have read that if the SMB stack is updated via Macport the results are favorable but I have yet to read anyone doing that for the same reasons that I would be looking to use.

Oct 31, 2014 1:10 PM

Reply Helpful

Oct 31, 2014 1:33 PM in response to Bryan Frizzle In response to Bryan Frizzle

Acronis bought GroupLogic who make Extreme-Z IP which is an AFP file server for running on Windows servers. One of the features it provides is built-in Spotlight indexing so it can be searched by Macs.


While one would expect your solution to work equally well for Mavericks as well as Mountain Lion if you buy a commercial solution like this you should get the reassurance they have solved these problems for you.


See http://www.acronis.com/en-gb/mobility/mac-windows-compatibility/

Oct 31, 2014 1:33 PM

Reply Helpful

Dec 3, 2014 11:43 PM in response to Bryan Frizzle In response to Bryan Frizzle

Your situation is word-for-world the exact same issue myself, fellow graphic designers, and IT department are facing.

I have been aware of this problem for well over a year now, and have been in constant search of a proper, "actually works" solution. No such luck as of yet.
I have also been aware of the ExtremeZ-IP option others say works for them.
In favour of trying to remedy the problem with a non-paid solution first however, I searched the deepest depths of Google to no avail.
I have been in phone contact with an Apple IT rep, and to be honest, they are oblivious of the issue, or are ignoring the fact that this is a real issue.


In the move forward to advance with technology, we must, and have upgraded OS X to Mavericks, and in the future, Yosemite.
The design department I work in has a handful of designers. We all use Macs. Our workflow is setup so that we save all of our work, Photoshop, After Effects, Cinema 4D, and any other graphic files are saved on a central file server so that other designers can access, and edit each others files at any time from any Mac. We work in a live television news environment, so searching of these files in a lightning fast manner is absolutely crucial.


Before this fiasco, the Macs ran on OS X 10.6.8. The Windows server holding our files is running Windows NT 4.
With OS X 10.6.8, doing a spotlight search for filenames of previously saved graphic files was quick, reliable and well, just worked.

Since the upgrade to Mavericks, this new SMBv2 protocol is what I believe to be the culprit of this evil. In fact, searching is just half the battle now.
ANY server interaction it seems makes the Mac slow, unresponsive, and sometimes F's it up so badly you have to push the power button!
For example. In Photoshop, naving to a folder on the server to save a file, it could sometimes take 5 or more minutes for the folders to even populate, and then sometimes just as long to save the file.


The problem with ALL of the solutions others mention is that they are missing the point. You should NOT need a Finder replacement. Finder, is a core, essential part of OS X. (Imagine Windows without Windows Explorer!) Finder must and SHOULD "just work". Finder replacements like EasyFind only search in list view. Designers search visually - aka, thumbnail view. Finder Replacements also do not replace File-Save or File-Open windows in apps, like Photoshop, or AI, AE or anything else. There is no integration that way.


So, recently speaking to the IT people managing our Macs, it looks like we might perhaps opt for a local, Mac-friendly server option.


I can't even to begin to describe the frustration this has cause us. This has single-handedly changed my perception of Apple, and their view of enterprise support and creative professionals. It reached a point where I said, I'd be happy to switch to a PC. And for me to say that....


So, Apple, if you read these boards, know that you have let a huge chunk of your user base down. Royally.


it's nearing 2015. It's time to "Think Different".

Dec 3, 2014 11:43 PM

Reply Helpful (1)

Feb 3, 2015 7:36 AM in response to stefanfromeuskirchen In response to stefanfromeuskirchen

Yes, it's a God **** shame; that AppleCare Support, has no clou whatsoever.......what kind of problems; mixed-platform users on a daily-base have regarding (AFB) - SMB1/2/3 connecting, searching(spotlight) and using files!!

Upgraded to Yosemite and the Saga continues.......


Regarding compatibility Windows Server 2008R2 shares, AppleCare Support says:

Check compatibility at Vendor (***? Microsoft...)


Apple should make rock-solid protocol solutions and pro-activ help out customers!


Searching on SMB2 and AFP(OldSchool) is dead slow and cumbersome.


Does anyone know if Extreme-Z IP is the Holy Grail even with SMB-connected server shares with Yosemite Adobe Cloud users??

Feb 3, 2015 7:36 AM

Reply Helpful

Feb 3, 2015 8:55 AM in response to macfanatic12 In response to macfanatic12

The way ExtremeZ-IP works is that either -


  1. You install it on a Windows file server which by itself is an SMB type server and then ExtremeZ-IP firstly uses the built-in Windows indexing to create a Spotlight compatible index. ExtremeZ-IP then also creates a full-blown AFP server on the Windows server. It is the AFP share via ExtremeZ-IP that the Macs would access files.
  2. Or, you again install ExtremeZ-IP on a Windows server but this time the Windows server itself is not a file server, instead the Windows Server connects to a NAS device which is a generic SMB server. Once again ExtremeZ-IP would use the built-in Windows indexing to create a Spotlight compatible index, and ExtremeZ-IP would 're-share' the NAS via AFP to the Macs.


According to this article AFP is still faster. See http://arstechnica.com/apple/2014/11/a-power-users-guide-to-os-x-server-yosemite -edition/5/


However I thought I remembered seeing another article suggesting SMB3 might be slightly faster. You could setup a test yourself. Of course if your using ExtremeZ-IP you will in any case not be using a Mac server and therefore the situation maybe different. Again you might need to setup a test using the eval version of ExtremeZ-IP.

Feb 3, 2015 8:55 AM

Reply Helpful

Oct 7, 2015 9:34 AM in response to John Lockwood In response to John Lockwood

I am so frustrated trying to get indexing and searching working. I have actually put an apple server in place and I still cant seem to get searching to work. I have security and permissions working correctly, but I can't get searching to work. I can't even get file name searching to work. I am very new to Apple networking and this has been a major source of frustration...not being able to search full-text.


When I start searching from the finder and use the option to search Shared Drives, file name searching works, but not on my new Mac server. Content searching does not work on the non-Mac servers. But name searching seems to work. When I choose to search just the new Mac Server folders, no searching works.


Do I need to someone turn on indexing with some terminal command? I work with 5 attorneys and not having good full-text searching is really poor.

Oct 7, 2015 9:34 AM

Reply Helpful

Oct 8, 2015 9:44 AM in response to garyleefromsaintpaul In response to garyleefromsaintpaul

I am in the process of trying to get this working for a client. It had worked a while back, but now does not. I THINK that it might have to do with sharing over SMB vs AFP - the little bits of info I can find seem to indicate that it will only work over AFP


See here for example: https://apple.stackexchange.com/questions/164443/does-os-x-yosemite-support-spot light-on-network-drives


Tonight I might try turning off SMB on the Yosemite server (it currently does both SMB and AFP) and see if that gets it working.

Oct 8, 2015 9:44 AM

Reply Helpful (1)
User profile for user: stefanfromeuskirchen

Question: How can I make OSX index a network share for spotlight?