~/Library/Metadata/SpotlightKnowledgeEvents using a lot of disk space

I manage around 200 apple desktops with a mix of Intel & Apple processors of the computers I manage I have identified 43 computers with this issue all of which are intel, all of which are running 15.1 or later.

On my intel test computer the files have been being generated since Oct 28th which is the day that MacOS 15.1 was released (and I would have probably installed it on day 1 to test if doing so would cause issues for my users).

I feel this issue should be addressed and fixed by Apple

I didn't get any confirmation from Apple support (because I didn't contact them), but I didn't find any problems after deleting the files. Also, before deleting the files, I made a copy of the files I was deleting just in case. I also asked for advice from friends who have been working with Macs for a long time and have a lot of experience. Their answer was simple - just delete it and don't worry 😅🤷‍♂️

I will chime in.

I have a 2018 intel mini and an M2 MacBook air. The folders in issue have grown massively (as large as 60 GB on the intel, and I have deleted their contents a couple of times. Yesterday, I deleted the contents of SpotlightKnowledgeEvents and CoreSpotlight and updated to Sequoia 15.4. SpotlightKnowledgeEvents is now inside CoreSpotlight and has already grown to 9.11 GB. Almost all of it (9.1 GB) is inside .../SpotlightKnowledgeEvents/index.V2/journals. The subfolder number, previously 10 and then 11, is now 12.

On the M2, there has been no problem and I have never touched it - the entire corespotlight folder is 6.5 GB. Most notably, however, .../SpotlightKnowledgeEvents/index.V2/journals is only 12k on the M2.

So, it looks like there is work being done on this problem, and evidently some progress, but this accumulation of journal files appears to remain out of control.

I have a 2018 Intel Mac Mini running Sequoia 15.3 with the same problem - 137 GB of my 1 TB internal SSD is used by Spotlight - specifically

/Users/<my account>/Library/Metadata/SpotlightKnowledgeEvents/index.V2/journals

In that folder, I have 2 subfolders, 10 and 11, and each of those has two subfolders, cs_default and cs_priority:

Under 10, cs_default is 5 GB and cs_priority is 127 GB, and the files modification dates run from 10/9/24 to 11/20/24.

Under 11, cs_default is 117 GB and cs_priority is 14 GB, and the files modification dates run from 11/20/24 to now.

10/9/24 is when I installed 15.0.1 (from 14.7); 11/20/24 is when I installed 15.1.1.

So, I am thinking perhaps the 10 folder is obsolete and now longer used and can just be deleted, anyone have any ideas about this?

ericmurphysf wrote:

The advisor said that Apple okays this approach, with the caveat that you should delete these two folders' contents, not the folders themselves.

Which two folders did Apple say were OK to delete the contents of? Was it CoreSpotlight and SpotlightKnowledgeEvents, or was it cs_default and cs_priority? If the latter, it would mean 4 folders to delete.

Also, which folders are regrowing, is it all 4 folders in the 10 and 11 folders?

I'm also thinking if it will just repopulate everything fairly quickly, then necessitating another manual pruning, this cycle would also decrease the SSD lifespan. I am hoping this is a bug and will be fixed, although as it seems exacerbated on Intel it may be low priority. I am moving to AS soon, but don't want to bring over 100 GB of possibly unnecessary cruft with me (too lazy to do a clean install, I admit it).

In the mean time the following script run when a user logs in (or set to run as a LaunchAgent) will delete the offending directory whenever a user logs out.

As these files belong to and can be deleted by a non privileged user account I doubt it cause any issues to the underling OS, examining what was logged on a computer that is set up to do nothing other then record a webcam what is being logged in these files is total rubbish. (in its case information about tuning your guitar to match the instruments in garage band)

But this needs to be fixed by Apple as it will in time result in every intel apple computer running MacOS 15 grinding to a halt with a full disk.

#!/bin/sh

onLogout() {
    rm -r ~/Library/Metadata/SpotlightKnowledgeEvents/index.V2
    exit
}

trap 'onLogout' SIGINT SIGHUP SIGTERM

while true; do
    sleep 86400 &
    wait $!
done

exit 0

Hello everyone.

A few days ago I discovered the same issue. I use a MBP 2019 with an Intel chip, 256 GB, Sequoia 15.3.1 and I received a message about a low free disk space. You should have seen my face when I found out that more than 150 GB of disk space was occupied by system data 😳. I am not a pro-user in the macOS system, and Apple claims that system data is a closed section with limited access. So it took me a few days and a lot of internet searching to find the problem. And in my case - the reason is in the section: /Users/***/Library/Metadata/SpotlightKnowledgeEvents/index.V2/journals/11/cs_default

There are files with the extension *.journal with a total volume of over 100 GB accumulated there. I opened the largest of them in a text editor and according to the content - they are related to the Microsoft Remote Desktop program - this is the program that I currently work with the most time. After reading the comments here on this topic - various assumptions are made regarding the programs that can lead to the accumulation of excess files. I assume that the problem is at the macOS level, and the largest files relate to those programs that you work with the longest during the working day

Still same for me, OS 15.4.1, 2018 Mac mini (Intel), path ~/library/metadata/CoreSpotlight/SpotlightKnowledgeEvents/index.V2/journals/12/cs_default this folder consume rest of free space (about 70GB), so once a week I need to delete all files in that folder.

I have disabled Spotlight indexing of the entire hard drive, disabled all indexing options, and restarted the computer. Files are still being generated but at the moment they are small (a few KB each file).

I have reviewed some large files (some are over 1.3GB), and indeed it seems to contain email indexing information.

I have a couple of mailboxes that are quite large, but the information in this folder only looks like a "journal" information of indexing process (as it includes in the file names), and it is not properly deleted.

I'll restart Spotlight indexing again, and we'll see what happens, but I think I'll eventually delete the files manually

XRoom wrote:

I have disabled Spotlight indexing of the entire hard drive, disabled all indexing options, and restarted the computer. Files are still being generated but at the moment they are small (a few KB each file).

I have reviewed some large files (some are over 1.3GB), and indeed it seems to contain email indexing information.

I have a couple of mailboxes that are quite large, but the information in this folder only looks like a "journal" information of indexing process (as it includes in the file names), and it is not properly deleted.

I'll restart Spotlight indexing again, and we'll see what happens, but I think I'll eventually delete the files manually

I tried reindexing on all four of my Macs (twice on my iMac Pro), and oddly enough none of the existing Spotlight metadata seems to have been deleted during or after the reindex. I still see Spotlight metadata files going all the way back to when I installed Sequoia on each of these systems.

I rectify, really after the restart it has created large files, some are more than 1GB.

I had only looked at the .toc type files when the big files are really the .journal type ones.

The content of some files I've reviewed points to email indexing, even though indexing for everything is disabled.

It clearly seems like a BUG in this version of Sequoia, although it does not occur on all systems (there may be some other factor causing it)

ericmurphysf wrote:

XRoom wrote:

I have disabled Spotlight indexing of the entire hard drive, disabled all indexing options, and restarted the computer. Files are still being generated but at the moment they are small (a few KB each file).

I have reviewed some large files (some are over 1.3GB), and indeed it seems to contain email indexing information.

I have a couple of mailboxes that are quite large, but the information in this folder only looks like a "journal" information of indexing process (as it includes in the file names), and it is not properly deleted.

I'll restart Spotlight indexing again, and we'll see what happens, but I think I'll eventually delete the files manually

I tried reindexing on all four of my Macs (twice on my iMac Pro), and oddly enough none of the existing Spotlight metadata seems to have been deleted during or after the reindex. I still see Spotlight metadata files going all the way back to when I installed Sequoia on each of these systems.

I’m not sure what the folder in the topic of this discussion holds, but re-indexing Spotlight only alters the data stored in the hidden .Spotlight–V100 folder at the root of the drive.

essentially, all metadata attached to a file is Spotlight metadata, but it is not removed when you re-index spotlight.

I did some further experimenting today, given the SpotlightKnowledgeEvents folders on my two Intel Macs continue to grow at as much as 10 GB/day (which will exhaust the free space on these systems in roughly two months).

This morning I explicitly excluded the folder where my large Pages files reside in the filesystem (Spotlight seems to spend most of its time indexing large Pages files). But even after excluding this folder, I still see .journal files in the cs_priority folder that make repeated references to the specific file from this folder I have open in Pages. It seems that even explicitly excluding a folder from Spotlight search does not prevent Spotlight from continuing to write .journal files to the SpotlightKnowledgeEvents folder.

I have an open ticket (actually, two of them, along with another ticket with what I believe to be a related issue with Time Machine) on this issue, and one question I would like answered is if I can simply delete all of this metadata. So far no one at Apple has been able to give me an unambiguous answer to this question. It seems unreasonable, to put it mildly, that Spotlight would produce five to ten gigabytes of metadata files mainly concerned with a single 12 MB Pages file.

Question: Is deleting .Spotlight-V100 necessary as well? Anyone has experience with deleting .Spotlight-V100?

I have no idea why it would help in this situation, but I've deleted it many times, once on an external to solve a problem with searching that drive, and several other times in the startup drive in order to be able to recommend it as a possible solution to some other Spotlight problems. You can also remove it in Terminal:

sudo mdutil -X /
sudo mdutil -i on /
sudo mdutil -E /

The first may disable indexing, so the other two are to kickstart the indexing process.

FWIW, the path is actually: ~/Library/Metadata/CoreSpotlight/SpotlightKnowledgeEvents.

Mine is currently 80GBs so 150GBs is not particularly out of the realm of possibilities.

You could start in Safe Mode which will clear some caches, reboot normally and see if it is smaller.

Start up your Mac in safe mode - Apple Support

Spotlight also indexes emails and if you have a huge number of emails that will increase the index size. Does it seem like you would have an extreme amount of emails, maybe over several email addresses totaling more than a thousand?

I do agree that the files size you noted seems like a lot compared to what I see on my device and reported by dialabrain.