Continued corespotlightd process CPU overload issues

Hi everyone. I also encountered this issue that corespotlightd was slugging down my M1 MBP 16GB (2021) so immensely that my system had a freeze for around 5-8 seconds every minute or so.

Reading that according to your findings it might be related to large Pages files it got my attention because I'm currently working on my Thesis and use Zotero with lots of indexing and caching. I assumed this might be the limit of this machine but that thought was strange because I worked on so much more taxing tasks and it just performed good enough that the operating system was still performant enough. My Thesis file currently only has half a MB (currently mainly text) so that can't be the issue I thought.

After working for days like this (it really gets frustrating) I decided to invest some time in troubleshooting again. Before that I tried to reindex Spotlight (through System Settings and Terminal) or cleared up some space but nothing did the trick. Also not even turning off Apple Intelligence which I thought could be the culprit made a difference. Until I stumbled upon some thread somewhere which just generally stated that deleting the Cache Folder in Library (Finder>Go>Go To Folder>~/Library/Caches) might help or not but it's generally not a bad idea to clean it out from time to time. Well I didn't do that for like 4 years! Which actually speaks for the rigidity of macOS.

I went to that folder and it had a size about 50GB and literally right after deleting it the freezes and the high CPU usage of corespotlightd went away. I now waited several hours to see if it was just something temporary but it seems like this was indeed the solution.

And I forgot to mention: I upgraded from 15.2 to 15.3 several days ago and it seems like something in the Cache became corrupted or faulty (be it system files or app files) and caused corespotlightd to go rampant.

So in short: give the cleanup of the ~/Library/Caches folder a try. It might help and solve this high CPU usage of corespotlightd. Hope this helps anyone.

After the better part of another day thinking about and troubleshooting this issue, I am convinced that Eric Murphy's earlier hypothesis is correct. There's a bug in Sequoia, which anyone can replicate by following these 2 steps:

1. Open a Pages file (and keep it open).
2. Watch the size of this folder balloon: ~/Library/Metadata/Corespotlight

The larger that folder gets, the more likely it is that the corespotlightd process will start taking over the CPU and causing slowdowns for the Mac user. The corespotlightd process is what gets most people's attention, but it's only a symptom of the underlying problem whereby the spotlight processes (mdworker, etc.) write enormous amounts of data into the corespotlight subfolders.

The bigger the Pages file the quicker the folder grows in size; the more frequently one uses Pages, or leaves Pages files open, the worse the problem.

There is no fix until apple implements one, and the only viable workaround is to monitor the size of that folder and occasionally delete it.

One silver lining: it's not clear to me that there is any need to delete your spotlight index, to turn indexing off and on, etc. The problem stems from the size of that metadata folder, and you can alleviate the problem by deleting the folder. In my experience (having deleted the folder many dozen times), spotlight works just fine without rebooting, reindexing, or anything else.

I came up with my own way of dealing with this issue: I wrote a simple shell script that trashes the corespotlightfolder; then I added that as a service in launchd so that it can run regularly (maybe every 2 days).

Mitch,

I think you could be onto something here. My brand-new, maxed-out Mac Mini was working perfectly, then after a kernel panic, reboot, and relaunch of all my open apps, corespotlightd started dominating the CPU and causing cursor and data input freezes. Pages.app has an iCloud-resident, password-protected file of text and images that I add to daily and that relaunched as a part of the system reboot. That file is currently 29.5 MB.

After reading your post I closed the large Pages file and after a few minutes corespotlightd dropped off the top of the CPU list in Activity Monitor. I did not delete any SpotlightResources.plist files. When I reopened the large file, corespotlightd again started climbing to the top and intermittent cursor freezes reoccured.

I quit Pages.app and corespotlightd disappeared as did the freezes. Now I've relaunched the app and reopened the large file,. All is well, no cursor freezes and corespotlightd is at 0% of CPU.

Looking at Console.app, I see this in one of the diagnostic reports (date and time concurrent with the freezing issue).

Command:          corespotlightd
Path:             /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Support/corespotlightd
Codesigning ID:   com.apple.corespotlightd
Resource Coalition: "com.apple.corespotlightd"(849)
On Behalf Of:     445 samples Pages [971] (445 samples originated by Pages [971])
Architecture:     arm64e
Parent:           launchd [1]
PID:              4063

I don't know if any of that is relevant. I speculate (wildly) that spotlight might have been trying to index an open, yet password-protected large Pages file and that was causing the system-wide issues.

The file has been open for 30 minutes and there are no issues and corespotlightd is not showing up on the CPU list.

I hope this helps.

Mitch, I can reliably reproduce the bug by opening a large (34 MB) Pages file, adding some text then letting the application sleep (TOP command in Terminal confirmed). After several minutes, corespotlightd takes over the CPU resulting in stuttered data input and spaces swaps, and cursor freezes. After quitting Pages the issue resolves after a few minutes. If you're going to open a bug report, I'm happy to submit configuration data and log files.

So … a couple of updates:

1). Talked to Apple Support yesterday. They took lots of notes, and have passed the issue on to MacOS engineering.

2). It’s been 3 days since I turned off Apple Intelligence on my iMac, and it has returned to normal operation - it took about 24 hours for things to settle down (I was seeing spikes in corespotlightd, but it would come down off them). Since then it’s been behaving itself.

3). Regarding the system CPU usage spikes, I was seeing some spikes of processes like kernel_task when things got into a really hairy state. I suspect this is related to resource allocation issues where corespotlightd may have been demanding exclusive access to particular resources and then getting into the weeds. (This would also explain the escalating stuck process count in top, now that I think about it).

I'm hoping to talk to yet another Apple senior advisor tomorrow (Saturday Feb. 15), not so much seeking a resolution for this issue as to provide Apple's engineers with as much detail about this issue as I can gather.

In pursuit of that goal, I have a question: is anyone seeing these large accumulations of metadata in the ~/library/metadata/CoreSpotlight/ and ~/library/metadata/SpotlightKnowledgeEvents/ folders (the latter is a separate folder on Intel systems) without using Pages?

In my experience, if I don't have a Pages document open, Spotlight metadata generally does not grow by more than a few hundred megabytes a day (which ain't nothing, but it's a lot less than the 10 GB/day or so I see if Pages is open). I'm curious if anyone has seen large increases in metadata despite not having Pages in use.

I, too, have been suffering with this issue on 4 different Macs, and I've been testing a million different things. I've also read this entire thread. Just trying to add a few summaries here that specify what we know and don't know.

But before I do that, I just have to say that Apple's "sort by rank" default option is a nightmare: it just makes the thread seem like meaningless nonsense, even though so many here are trying hard to solve the problem together.

1. I think Eric Murphy has shown that the corespotlightd problem (cpu usage spiking) is itself an effect of the metadata folder blowing up.
2. This means that if you delete the metadata folder, you will receive temporary relief. Same thing goes for having spotlight reindex, and perhaps even turning off AI – these things are all temporary.
3. CaptainJoy raises a very important question: how many people having problems are working on systems with migrated data?
4. Second question: is anyone having problems with optimize iCloud Drive turned ON?

I have 2 M2 Pros that are having exactly the problems described here, both had migrated data at one point in the past, and both have optimize iCloud Drive turned OFF.

But I have an M3 MacBook Air that was set up this summer from scratch. It downloaded iCloud Drive data, but I migrated nothing. It has optimize iCloud Drive turned ON. That computer is the only one not having problems. The metadata folder is less than 5Gbs and while it's growing, it's very very slow.

My guess is that optimize iCloud Drive is not the underlying cause, it's just that with that turned on, spotlight has less to index and therefore makes less of a mess.

But I really wonder if anyone out there can replicate this problem on a brand new Mac without any migrated data???

AshkaTheMoltenFury wrote:

So in short: give the cleanup of the ~/Library/Caches folder a try. It might help and solve this high CPU usage of corespotlightd. Hope this helps anyone.

Unfortunately, I think that apple's default to "sort by rank" means that many people are MISSING the most important discoveries in this thread. The above WILL make things better, but only temporarily – it's treating the symptom, not the cause.

The cause of all this, as ericmurphy has laid out and a number of us have replicated, has to do with a problem with spotlight indexing of Pages files. Even if you clean everything out, as above, if you then open Pages files (especially larger ones) and keep them open, you can literally watch as the various mdworker processes write MASSIVE amounts of data into the core spotlight metadata folders. Depending on other aspects of your system, at some point that folder will get so big that the corespotlightd process will slow your Mac down.

• The temporary workaround is to regularly delete the metadata folders.
• The temporary and still very much less than ideal "fix" is to TURN OFF spotlight indexing.
• Any real solution here will require Apple to make some tweak to spotlight or Pages.

I did not have quite the same problem.

In my case Spotlight (mds_stores) was writing up to 100 GB per day and apparently deleting it instantly as nothing ever appeared on my SSD and there were no unpleasant symptoms.

However, it was causing massive unnecessary writes so I selected all my drives in System Settings/Spotlight/Search Privacy and the writing has completely ceased. Presumably all other Spotlight activity has stopped.

Luckily I can live without Spotlight.

On my M4, tried

while true; do killall -9 corespotlightd 2>/dev/null && sleep 0.5; done &

this seemed to get rid of the process if run for a few seconds. But then opendirectoryd comes up and consistently uses about 20% of cpu.

Offering some additional info that may or may not be useful.

When I was experiencing this issue (before updating from 15.1.1. to 15.2), there were two very large store.db files in the meta data:

Library/Metadata/CoreSpotlight/Priority/index.spotlightV3/store.db

Library/Metadata/CoreSpotlight/NSFileProtectionCompleteUntilFirstUserAuthentication/index.spotlightV3/store.db

Both were over 11GB each.

Since updating, when I'm working with iCloud-based Pages files, although corespotlightd occasionally shows on Activity Monitor taking over 300% of CPU, it doesn't stay long or create any persistently stuck processes, cursor freezes, or data entry issues. However, those store.db files have now been reduced to 24MB and 193MB respectively. From GB to MB has got be an improvement, right?

MC2G wrote:

I have checket and I also have a store.db file of 17,51 Gb and another one of 1,91 Gb.

What soul I do about it? Can I just delete them?

I'm not sure what you mean by "store.db" files. What I did was destroy the contents of my CoreSpotlight and SpotlightKnowledgeEvents files (but not deleting the enclosing folder itself), as suggested by ericmurphysf. It's been about 18 hours since I did that and the fix has held.

I will add to this discussion the following webpage that a friend of mine hipped me to; it seems to corroborate ericmurphysf's advice: https://www.spyhunter.com/shm/fix-macos-sequoia-spotlight-issues/. Yes, they have software to sell, but they advocate deleting the whole CoreSpotlight folder (which also contains the SpotlightKnowledgeEvents on new computers) and deleting another Spotlight folder in /System/Volumes/Data/.

Now that my Time Machine backups are working again, I will probably try the above the next time the corespotlightd process gets too big for its breeches. After less than a day, it's up to 21.41 GB. I wish I noted how big it was before I cleaned it out. Going forward, I request people let us know how big their CoreSpotlight folders are before they delete them.