Continued corespotlightd process CPU overload issues

Zenith wrote:

Next stop on the troubleshooting train: exclude ~/Library/Metadata from spotlight. Why would we need spotlight to index that anyway for routine operation?

I think you're right about what is causing Spotlight metadata to balloon (and have been discussing the matter with Apple support for over a month; currently I'm waiting for a response from engineering).

That said, I tried your troubleshooting step noted above, with negative results: Spotlight metadata continued to grow, and much more quickly if Pages is running with a document open. The only "fix" for this issue I've found to work so far is to regularly delete Spotlight metadata out of the ~/library folder. I'm a pretty heavy user of Pages, so I have to do this about every ten days, by which time metadata has usually exceeded 60 GB.

An update on my status. Similar problems to everyone else. I too have large pages files, several machines, and iCloud file storage and sync. Deleted spotlight metadata a few times. Helped for a while, then the problems returned -- large metadata folders, high CPU load, and that weird rhythmic spiking of the CPU usage.

This week, the mouse (old Logitech with usb dongle for bluetooth) lag was significant across several apps, beginning with pages. Thought my mouse batteries were the problem. Changed batteries. Didn't work. Changed mice - went through all 3 of my usb dongle-bluetooth Logitech mice, and then switched to my bluetooth only laptop mouse. The problems went away.

Got new bluetooth only mice, and also dumped the metadata again. CPU load has been low and steady, Corespotlight metadata folders have stayed under 10 GB for a week!

Yet another weird data point.

ericmurphysf wrote:

Mitch Stone wrote:

No evidence points to iCloud being implicated in this issue in any way whatsoever. Certain large Pages documents that have been extensively edited appear to be the most common trigger of this issue, and it is not dependent on them being stored locally or in the cloud.

From what I can tell, what triggers rapid accumulation of Spotlight data specifically with large Pages documents is the way macOS automatically does versioning of such documents. That versioning is not due to sync via iCloud; it's simply the design decision of allowing the user to revert to any prior version of the doc, or indeed copying data from earlier versions to the current version if e.g. you accidentally deleted a paragraph or section of a document.

My hypothesis (so far not confirmed by Apple) is that the rapid growth of Spotlight metadata is an artifact of the various daemons doing Spotlight indexing (md_worker, etc.) reindexing the entire document, rather than just newly-edited sections. This happens whether or not you have iCloud sync turned on.

I obviously can't confirm it either (we need apple for that), but in my many hours of troubleshooting this, I am always led to the conclusion that the above hypothesis is exactly right.

I too have mentioned it happens due to Numbers. Having had lots of occurrences and problems when using Pages, I thought I had the same issue with Numbers. I made sure that neither Pages nor Numbers was running, I think I restarted, and as soon as I started using Numbers problems, started.

However, it might be limited - as in only some spreadsheets. I am doing lots of more-or-less "text database" work in Numbers - rather than calculating. Possibly in numeric/calculating Numbers sheets there isn't a problem? (Just wanted to say this so that others can check this before replying.)

I agree with Mitch Stone. I have experienced the corespotlightd overload on my Mac mini M4 OS 15.5. However, as long as I turn off Pages when I’m not using it, especially when working on a very long document, and the Mail.app as well, I’ve seen the Core Spotlight folder diminish from 30 GB to as low as 4 GB. It tends to grow again but never beyond 30 GB, at least so far. I have not turned off Apple Intelligence. Along with Mitch, I would not mess around any more with deleting Core Spotlight files and the like. My Mac runs smoothly and no longer overheats as long as I follow these precautions.

I agree - think that there is indeed an iCloud link.

In particular, I have more recently been much more careful about making sure I do not have the same documents open on both my macOS machines.

That, combined with changes I suspect fed through in updates in the past few months, has improved my actual experience. I can work on my machines without beachballs appearing repeatedly.

Nonetheless, I think the CoreSpotlight folder has grown excessively. Over 44 GB for 3646 items in my M4 mini. And about half that for twice as many document on my M1 MBP.

A pain because I keep jumping between machines - one upstairs, one downstairs. For no positive reason - I tend to just use whichever is closest when I decide to do something. But this means I have to think about things and make sure I close Pages and Numbers when I stop actively working. Otherwise I likely start again on the other machine...

I, too, have been suffering with this issue on 4 different Macs, and I've been testing a million different things. I've also read this entire thread. Just trying to add a few summaries here that specify what we know and don't know.

But before I do that, I just have to say that Apple's "sort by rank" default option is a nightmare: it just makes the thread seem like meaningless nonsense, even though so many here are trying hard to solve the problem together.

1. I think Eric Murphy has shown that the corespotlightd problem (cpu usage spiking) is itself an effect of the metadata folder blowing up.
2. This means that if you delete the metadata folder, you will receive temporary relief. Same thing goes for having spotlight reindex, and perhaps even turning off AI – these things are all temporary.
3. CaptainJoy raises a very important question: how many people having problems are working on systems with migrated data?
4. Second question: is anyone having problems with optimize iCloud Drive turned ON?

I have 2 M2 Pros that are having exactly the problems described here, both had migrated data at one point in the past, and both have optimize iCloud Drive turned OFF.

But I have an M3 MacBook Air that was set up this summer from scratch. It downloaded iCloud Drive data, but I migrated nothing. It has optimize iCloud Drive turned ON. That computer is the only one not having problems. The metadata folder is less than 5Gbs and while it's growing, it's very very slow.

My guess is that optimize iCloud Drive is not the underlying cause, it's just that with that turned on, spotlight has less to index and therefore makes less of a mess.

But I really wonder if anyone out there can replicate this problem on a brand new Mac without any migrated data???

fronesis47 wrote:

The temporary workaround• is to regularly delete the metadata folders.

The temporary and still very much less than ideal "fix• " is to TURN OFF spotlight indexing.

Any real solution• here will require Apple to make some tweak to spotlight or Pages.

Replying to my own post to amend it: It looks to me like the temporary fix is NOT a fix at all. I just turned spotlight indexing OFF, then opened a pages file and watched the corespotlight metadata folder grow over 5Gb in less than an hour. Whatever process is writing to the metadata folder is not turned off when you turn off spotlight indexing.

This means the only options are to live with this, by occasionally deleting the metadata folders, deleting the .plist file, and perhaps turning indexing off and on – all while waiting and hoping that apple fixes the problem.

I agree with Mitch Stone: it would be good to know if other documents (such as Numbers files) cause the same problem, and to see what happens with a new user.

THANK YOU to sugarskyline for showing me how to change the default presentation of the message board.

roysch53 wrote:

Regarding the two large folders (NSFileProtectionCompleteUntilFirstUserAuthentication & Priority) under the ~Library/Metadata/CoreSpotlight folder; I don't have "Advanced Data Protection" turned on, so how is it that the NSFile... folders exist - is this part of Apple Intelligence? The NSFile... folders do not exist on my Intel iMac, only on the M2 laptop.

Has anybody removed (or renamed) these large directories, turned AI off and restarted - I just wondered if they get recreated.

Also, has Apple Support shed any light as yet?

The folder structure for Spotlight metadata under ~/library/metadata differs greatly between Intel and Apple Silicon systems. I have yet to delete metadata from either of the Apple Silicon systems I own, but based on postings by others, it sounds like it's safe to delete the entire CoreSpotlight/ folder. After you delete these folders, macOS will recreate them after five or ten minutes, and then they'll start to grow again. But so long as I don't leave Pages files open when I'm not editing them, they don't grow nearly as quickly.

As for Apple Support: I've been dealing with them on this issue for the better part of a month now, and while they've been helpful, the only concrete information they've been able to provide so far is that it is safe to delete the Spotlight metadata folders. I've done Capture Data sessions on an Intel Mac and an Apple Silicon Mac, and also sent them a detailed email describing my research and findings thus far. Unfortunately, the last senior advisor I spoke to, the weekend before last, was going on vacation for a week. I'll follow up with him tomorrow and see if he has anything to add, although what I'd really like to hear is that the Spotlight team is working on a fix to be added to a future system software update.

Mitch Stone wrote:

Did you try my suggestion? For me, it actually did help a lot. I'm not sure I understand the resistance to it, given the simplicity and apparent effectiveness. I'd really like to know if it works as well for others as it did for me.

Based on the totality of posts on this thread, it seems very likely that while using Pages is not the sole cause of corespotlightd (and Spotlight metadata) running amok, it seems to be possibly the largest contributor. So far, users seem to have had decent luck with one, two, or all three of the following:

1. Mitch's recommendation of duplicating a Pages file (thus removing versioning information, which seems to be what spikes corespotlightd activity and Spotlight metadata accumulation), and then making further edits to the duplicated version.
2. Removing Spotlight-related .plists.
3. Removing Corespotlight metadata from the ~/library/metadata folder(s)

None of these workarounds are permanent fixes. But they seem to buy at least temporary reprieve (I deleted Spotlight metadata yesterday afternoon, and right now the system fan is at 12%, system CPU usage is below 20%, and corespotlightd doesn't even appear in the top forty processes). If I have to delete metadata once a week or so (a task that takes all of ten seconds), that's a small price to pay for acceptable performance.

If you're running into performance issues, including Time Machine issues, I would recommend you try some or all of these fixes. Apple may or may not ever get around to addressing these issues (which seem to disproportionately affect Intel systems which are obviously not Apple's priorities these days). But these fixes, singly or in some combination, seem to work for a lot of people. I'd at least give them a try. None of them seem to have negative consequences, unless you're unusually dependent on versioning in Pages.

Last week, I called Apple Support and described the corespotlightd issue on my 4-months-old M4 MacBook Air [MacOS 15.5]. An appointment was made with the Genius Bar at my local Apple Store.

At this appointment, I explained our Pages [v14.4] corespotlightd issue to a lovely chap and we ran the hardware check which passed, ruling out my machine. I showed the Genius this forum, and he arranged for next-level-up Apple Support to call me the next day.

On that call, I shared my screen and showed the (also lovely) support chap this forum. After creating a New User (to rule out a user-specific problem), I opened Activity Monitor, highlighted corespotlightd, created a Pages document and saved it to iCloud. After a few edits, corespotlightd spiked. 

The support chap then consulted and I was put on to a new (equally lovely, even higher-up) support chap to whom I showed this forum. He wanted me to duplicate the problem whilst screen recording. Unfortunately, I was unable to duplicate the spiking so the he sent me an email that allows me to send him a screen recording of the issue, should I be able to get it to happen.

I have been trying each day to capture the problem, but have, so far, failed. I will keep trying every day.

So we know that several individuals in Apple’s support hierarchy are at least  aware our corespotlightd spiking issue. How high in that hierarchy, I cannot say.

Perhaps, if a few more of us follow the Apple Support approach, we may be able to get this issue acknowledged and then solved.

I just bought a MacBook Air 2022 M2 with 16GB, after the immediate update it runs now Tahoe 26.1. I face the same problem, with the corespotlightd slowing down my processor, consuming energy, and stopping Pages from working every minute or so for a few seconds. I transferred the two Pages documents I was working on on my SSD, and miracle. My CPU load is below 10%, the machine is fast with corespotlightd at 0.1% ! I have Key, Numbers, Safari (a full load of tags open), Preview, Mail… open, and no troubles.

SO : it's not the system, it's just Pages.

Why is this not fixed ? please "Apple Guys" if you can't read that, ask an İ.A. to read the complaints and search the codes, or do your job, that shouldn't be so complicated !!!