Continued corespotlightd process CPU overload issues

CaptainJoy wrote:

tl;dr — trash your ~/Library/Metadata/CoreSpotlight folder

This has consistently been my experience. I start running into issues with Spotlight Search when this folder exceeds about 40 GB. The system runs into serious performance issues on my computers (which have 32+ GB RAM) when it gets above about 100 GB.

I'm a heavy Pages user, with 10+ MB Pages files open all the time, which seems to accelerate the growth of this folder. So once a week or so I just trash the entire folder.

It's kind of like mowing the lawn.

In my case the issue was related to Docker Desktop that I installed on my Intel-based MacBook Pro running Sequoia 15.7.1. I found more than 40GB of data in my ~/Library/Containers/com.docker.docker/Data folder. After uninstalling Docker App and clearing docker Data folder the issue with corespotlightd using a lot of CPU went away within few seconds.

I am not certain that I installed and used Docker Desktop correctly, therefore this is not a judgement on Docker Desktop (I find Docker very useful). Also, I uninstalled Docker Desktop and cleared the Data folder at the same time, so I am not sure if the issue was not related to Docker Daemon activity. However, inspecting ~/Library folders may help in troubleshooting the problem discussed in this thread.

I was having this problem with my M4 Mac Mini on Tahoe 26.2. I tried clearing the caches, Spotlight metadata folder, and even reinstalling the OS. Nothing worked.

The solution in my case involved deleting an old and relatively large Pages file stored in iCloud. I had been maintaining the document for years, so it had amassed a very long version history. Because I still needed the document, I made a completely fresh Pages document (not a copy of any other document) and moved the contents of the old document to the new. I then deleted the old one, and this seems to have fixed my problem.

If you have any Pages files on which you've made many edits, I recommend copying its contents over to a fresh file and deleting the old one. It wouldn't surprise me if this is also the case with Numbers, Keynote, or any file stored in iCloud which is capable of having a long version history.

As @mphollis said in the highlighted reply, I think files with extensive version histories are the culprit. Something about all that metadata sends corespotlightd on the fritz. Apple, please solve this problem!

Mitch,

I think you could be onto something here. My brand-new, maxed-out Mac Mini was working perfectly, then after a kernel panic, reboot, and relaunch of all my open apps, corespotlightd started dominating the CPU and causing cursor and data input freezes. Pages.app has an iCloud-resident, password-protected file of text and images that I add to daily and that relaunched as a part of the system reboot. That file is currently 29.5 MB.

After reading your post I closed the large Pages file and after a few minutes corespotlightd dropped off the top of the CPU list in Activity Monitor. I did not delete any SpotlightResources.plist files. When I reopened the large file, corespotlightd again started climbing to the top and intermittent cursor freezes reoccured.

I quit Pages.app and corespotlightd disappeared as did the freezes. Now I've relaunched the app and reopened the large file,. All is well, no cursor freezes and corespotlightd is at 0% of CPU.

Looking at Console.app, I see this in one of the diagnostic reports (date and time concurrent with the freezing issue).

Command:          corespotlightd
Path:             /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Support/corespotlightd
Codesigning ID:   com.apple.corespotlightd
Resource Coalition: "com.apple.corespotlightd"(849)
On Behalf Of:     445 samples Pages [971] (445 samples originated by Pages [971])
Architecture:     arm64e
Parent:           launchd [1]
PID:              4063

I don't know if any of that is relevant. I speculate (wildly) that spotlight might have been trying to index an open, yet password-protected large Pages file and that was causing the system-wide issues.

The file has been open for 30 minutes and there are no issues and corespotlightd is not showing up on the CPU list.

I hope this helps.

Mitch, I can reliably reproduce the bug by opening a large (34 MB) Pages file, adding some text then letting the application sleep (TOP command in Terminal confirmed). After several minutes, corespotlightd takes over the CPU resulting in stuttered data input and spaces swaps, and cursor freezes. After quitting Pages the issue resolves after a few minutes. If you're going to open a bug report, I'm happy to submit configuration data and log files.

I'm not sure how many other people with this issue have seen degraded Spotlight results (including in Mail) as a result of corespotlightd's misbehavior, but I managed to at least temporarily resolve some of these issues by, on the advise (or at least consent) of Apple support, deleting the contents of the two folders, CoreSpotlight and SpotlightKnowledgeEvents, from ~/library/metadata/.

Note that I deleted the contents of these two folders, not the folders themselves. Also note that on Apple Silicon systems the SpotlightKnowledgeEvents folder is inside the CoreSpotlight folder. On Intel systems, it's at the root of ~/library metadata.

However, deleting the contents of these folders (on my system those contents comprised over half a terabyte of data) did not permanently resolve the issue. In barely twelve hours Spotlight added 22 GB of new metadata to these two folders. But I think until Apple resolves this issue (I doubt it will be in 15.3), simply deleting the contents of these folders when they get over a couple of hundred GB will definitely improve system performance, especially search.

Also note that in my experience these issues are less serious on Apple Silicon Macs. On my M2 Max MBP and my M1 Ultra Mac Studio, these folders are large but not enormous; 40 GB on the first system and 18GB on the second one.

My 2024 M4 Mac Mini was unusable: spinning beachballs, screen locking up for a few seconds. I tried several things—force quitting the corespotlightd process, Rebuild the Spotlight index on your Mac, turning off Apple Intelligence, deselecting all categories from Spotlight search—none of this worked.

I can confirm, that as instructed by ericmurphysf above, when I deleted the metadata out of the ~/library/metadata/Corespotlight and ~/library/metadata/SpotlightKnowledgeEvents (while leaving the folders themselves intact), I was rewarded with near-immediate improvement. Hopefully this holds up.

I'm wondering why I was affected and others not? One unusual thing about me is that I was upgrading from a 2014 Mac Mini. Maybe the jump from Monterey (OS 12) to Sequoia (OS 15) when migrating my old stuff over via Time Machine had something to do with it?

sugarskyline wrote:

The person I spoke to wouldn't add my case to their file unless I tried booting in Safe Mode to see if the issue was still present, so be prepared for that, or potentially anything else disruptive for them to give your case validity. If you start with a chat that moves to the phone then also have your case number ready because the person on the phone won't have access to your chat log otherwise. The person I spoke to was fantastic so I wouldn't worry about dealing with typical poor customer service like you would from other companies. The call happened 5 minutes after the scheduled time and the chat representative showed up almost immediately.

One of the first things I tried, after this issue escalated so severely on my iMac Pro that Time Machine ceased working entirely, was to reinstall macOS (since I know from past experience Apple Support will frequently recommend doing so and will be reluctant to proceed further until you've done so). Unsurprisingly that had no effect on the issue, but Apple Support at least were willing to progress once I've told them I already tried that. If anyone does get such a suggestion I'd recommend you refer the advisor to this very thread, where numerous posters have stated that doing so has no effect on the problem.

Also, given the complexity of this issue, I would recommend that as soon as you get a support advisor on the phone, you request the issue be immediately be escalated to a senior advisor. No offense to Apple support advisors (who are excellent by comparison to the rest of the industry), but this is not the kind of issue a level I tech, who is mainly helping people install printers or replacing application icons that have been inadvertently dragged out of the Dock, are likely to be able to advise on.

Given what posters on this thread have determined so far, it's almost impossible to believe restarting in Safe Mode would have an effect. Doing so generally just disables non-macOS processes, but since the culprits are almost certainly one or more of the various Spotlight-related processes which are core to macOS and would still run under Safe Mode, doing so would largely be a time-wasting exercise.

Also, creating a new user account and using that might eventually be able to reproduce the problem, but since it seems like the metadata folders need to grow to at least 50 GB or more in size to be problematic, one would likely need to create a new user account, and then create/open/edit one or more Pages files, and wait a week or more (although some people report metadata growth rates that might bring that down to just a few days) to try to reproduce the problem.

Honestly, at this point I think Apple's engineers need to focus on how Spotlight indexes Pages files, and possibly other types of data files. At least in my experience (which admittedly does not seem to be universal), the core of the problem appears to be that Spotlight indexing processes will repeatedly reindex entire Pages files over and over again, appending those results to existing metadata rather than overwriting that data.

As evidence of that, I will again point to my experience over this last weekend. I had deleted the metadata from my 27-inch iMac on January 28, at which point it was 597 GB. From the 28th until February 7th it grew from zero to about 62 GB, with no ill effects. But then after leaving a single 145 kB Pages file open (but not being edited) from Friday evening until Sunday evening on a system that was completely idle, the metadata folders ballooned to over 120 GB, nearly doubling in 48 hours.

By contrast, with no Pages files open, this system would add about two gigabytes of metadata over a weekend.

Update: I agree the problem is primarily with Sequoia as I didn't have this at all until that update.

Since realizing the connection with Pages documents and the solution of emptying the ~/Library/Metadata/Corespotlight folder, I have only had to empty the folder twice. (about every 3 weeks of heavy use).

Key Points:

1. Turn OFF the Pages App when you are not using it. My Corespotlight folder WILL reduce in size when Pages is closed.
   1. Duplicate large/old Pages files in the finder and work with them. I've broken some documents into parts.
2. Empty the ~/Library/Metadata/Corespotlight folder when you notice slow down (on Activity Monitor or other).
3. Hope that Apple fixes the problem.

Thank you to those who gave safe solutions..

I can confirm that this works for me (MacBook Pro M3 14", Nov. 2023 + Sequoia 15.5)

1) Work on local Pages documents (temporary folder in Downloads for example).

2) Copy from time to time to iCloud folder such as Documents.

After a couple of days the size of ~/Library/Metadata/CoreSpotlight folder went down from 50 Gb to 0.7 Gb (!!!)

I had similar problems and posted a while ago about sluggish programs and overload issues with an M4 iMac. Per instructions from Apple 'elevated' support, I deleted the entire large folders in ~/Library/Metadata/CoreSpotlight/. I found that they rebuilt later but were smaller. It was not necessary to go into the folders and delete their contents.

I was also convinced that this was related to having a large Pages file open. But then, after archiving last month's 35MB+ file and starting a new, empty one, I had the same issues that would not go away, even if I quit Pages.app and waited. It was making my Mini virtually unusable for writing (my main task) so I switched to my iPad, rebooted the Mini and started running Clean my Mac routines on it. No significant change. corespotlightd had shot to the top of the usage list within a minute or so of reboot and stayed there. By this time, I had two terminal widows open alongside Activity Monitor and I was watching what was going on carefully ('-s 5' to slow things down and '-o state' to list stuck processes at the top). At various times I could have up to 20 stuck processes. One of them was launchd which led me to some further research and the idea of disconnecting my Time Machine Drive. Within a few minutes corespotlightd had vanished from the CPU usage list. Reopening the previously opened Pages file was fine until I swapped to another programme and then corespotlightd returned. I saved and quit Pages and it went away again. I'm including all this info in case anyone else is having the same issues as I am. Maybe try disconnecting the time machine drive? (It seems a bit nuts, I know Spotlight isn't supposed to index that drive.)

Mitch Stone wrote:

Interesting. Checking these folders on my M2, I find that the size of the CoreSpotlight folder is 37GB, but no file within it is even a megabyte in size. The SpotlightKnowlegeEvents folder clocks in at 463 MB. It contains far more subfolders, so it is difficult to figure out where this data is hiding. Has your performance been improved by deleting the contents of these folders?

And another FWIW: Since the last time I deleted the Spotlight plist about a week ago, I have had no corespotlightd process issues. So I do think this is worth trying.

As noted, this issue does not appear to be as severe on Apple Silicon systems as it is on Intel systems. My M2 Max MBP has about what yours has in the CoreSpotlight folder: about 42 GB. But many of the files in there are tens of megabytes, some over 100 MB (but I edit large Pages documents, and others have pointed out that this can exacerbate the problem).

So far I've deleted the contents of the referenced folders on the two Intel systems I own—a 2020 27-Inch iMac and an iMac Pro—and saw immediate performance gains, especially with anything having to do do with search: Spotlight searches, smart folders in Mail, etc. Corespotlightd also seems to have calmed down significantly, generally using less than 15% of available CPU time (yesterday I saw it go as high as 1,400% on an 8-core system). Prior to deleting this metadata, it could take upwards of five minutes just to log out of my account; now it's just however long it takes to quit all running apps).

I haven't tried deleting the spotlight .plists yet (there are appear to be five plist files with "spotlight" in the filename, at least on Intel systems), but I do see that Spotlight is still writing large amounts of metadata to the user library folder, 8.4 GB in just the last two hours. If that trend continues, I'l try deleting the .plist files as well.