why is photoanalysisd not scanning my library in Big Sur?

*Another Possible Solution*

After troubleshooting this for a looong time I discovered that the issue was with my specific library, not an issue with the system. Something to do with old caches, leftovers, and general garbage from having a large, old library. Here’s what worked for me:

PLEASE EITHER MAKE A DUPLICATE OF YOUR LIBRARY OR A BACKUP BEFORE DOING ANY OF THIS. I DID SCREW IT UP ONCE AND LOST ALL MY ALBUMS AND TAGS.

You want to first un-hide invisible files, go to the Terminal and type:

defaults write com.apple.finder AppleShowAllFiles -boolean true
killall Finder

Quit Photos.

In the Finder, select your library, right click and go to Show Package Contents.

You’ll see several folders, we’re going to focus on database, private, and resources.

In the database folder:

• Delete any hidden folders
• Open the search folder, delete any files inside
• Back in the main database folder, delete anything with a .db, .plist, or no extension
• IMPORTANT: leave anything with a .sqlite extension alone

In the private folder:

• You’ll see several folders called com.apple.something, delete the contents of these folders

In the resources folder:

• Delete the contents of the caches folder
• Delete the contents of the journals folder

Restart, open the library, quit Photos, and you should start to see some activity. My computer scanned ~20k photos in one day.

I forgot to mention one suggestion made by the Apple Support engineer (in their Creative Media division). They suggested creating a brand new user on my Mac and importing some photos to the (empty) Photos library there, with a view to identifying whether the issues I'm seeing are specific to my particular library or user, or something system-wide on my Mac.

The good news is, when I created this new user, imported around 50 photos and then left it scanning overnight (with the Photos app stopped), the photos now have circles around the faces.

While my specific issue is still not solved, this at least proves that the Photos app is not fundamentally broken on Big Sur. I'll report back again once I speak to the Apple engineer again.

Here are the changes I made on my system to fix this issue:

1) Remove Login Items

Click System Preferences, Users & Groups and select the Login Items tab for the Current User. Take a screen shot and then remove each item listed. I've since reinstated all of my settings here, rebooted, and re-imported some more photos and the faces were still detected, so I'm not sure this step is entirely necessary.

2) Malware Scan

I downloaded the free edition of Malwarebytes for Mac (https://www.malwarebytes.com/mac) and ran a scan on my system, fixing any issues it detected. The scan took under 5 mins to complete.

3) Configuration Reset

3a) For each of the directories listed below, do the following (if there are files therein)

Select all files/directories in the folder (Cmd-a) and select File, "New Folder with Section (x items)" from the Finder menu. This will move the selected files into a new sub-folder (called "New Folder With Items"), which is a process called "nesting".

3b) The directories I dis this for were:

User Settings

~/Library
~/Library/Caches
~/Library/Input Methods
~/Library/LaunchAgents
~/Library/Saved Application State

System Settings

/Library/Caches
/Library/Input Methods
/Library/LaunchAgents
/Library/LaunchDaemons
/Library/StartupItems

Once all of the above was done, reboot your Mac. Then, making an assumption that the Photos library updates that Big Sur had wanted to do in the first place never actually got going, leave the Photos app stopped for a few days.

My library is around 35GB in size and I think it took around 36 hours for the process to complete, so I wonder if 1 hour per GB (in library size) might work as a rough gauge here?

Best of luck!

And the volume is plugged into a USB or Thunderbolt port and not accessed over the network? And always plugged in to the Mac, before you sign into the user account?

Where are you seeing, that there is still plenty to scan? At the bottom of the People album?

If the face detection is hanging, you may want to check the videos and image files in your library for incompatible media items. In my library the face detection stopped because of some old videos, that were in a format that needed converting to a more compatible format. You may want to check this list of formats for legacy media, that should be avoided on Big Sur:

About incompatible media in iMovie for macOS - Apple Support

As a rule of thumb - any item that is giving an error message, when you try to import it into a Photos Library, should be removed from your Photos Library and replaced by converted items. I have checked all videos and older image files, by exporting the originals and trying to import them into a test library.

Issue FIXED!!!

Good news at last. With the help of the Apple Support engineers (from their Creative Media division), we managed to get face detection working again on my system.

*Troubleshooting*

Remember, from previous updates, we had proved that the issue was specific to my user account and not the Photos software or not something system-wide on my Mac. We did this by creating a new user, with a blank Photos library and a handful of imported photos, which detected the faces no problem. Whereas, three different Photos libraries in my main user account failed to detect any faces, confirming the issue was not specific to my main library but to all libraries in my user account.

*Our Diagnosis*

The Apple engineer then talked me through disabling a couple of items on my account and resetting some settings across the account (more details below). We also installed some Malware detection software, which detected 2 minor issues (corrected) and rebooted my Mac.

The Apple engineer then suggested that, it the settings of one of my other applications (or the Malware offenders) was somehow blocking the photo analysis daemon from running to completion, the important rescan that's necessary after upgrading to Big Sur (note: Big Sur includes Photos V6 which is a Major version update, with some significant library updates needed) would never really have gotten started. Therefore, they recommended waiting another couple of days to see if any progress would be made. It was!

Around 36 hours after we did the above, my library finally detected faces in. the imported apps again. I'm still not entirely sure which configuration setting or application was causing the issue, and don't yet know if some of my apps are adversely affected by the changes I made in their configs ... but I suspect I'll discover more on this in the coming days too.

Specifics on what I did coming in the next reply...

Thanks for taking the time to compose such a comprehensive update, putzfetzenORG. My two takeaways from your comments are:

1. I totally agree that some level of insight into, or control over, the facial scanning features would be hugely helpful. Even access to a log file somewhere that provides updates on what's actually going on (i.e. that helps explain the messages you see in the UI) would be enormously helpful, and may even reduce the number of support queries to Apple Support.
2. Not sure if you saw it, but the recommendation from Apple Support is that, in order for the faces/people scanning to progress, the definition of "in the background" actually means quitting/stopping the Photos app altogether.

Thanks again!

Thanks for all of the detailed suggestions from everyone on this thread! After upgrading directly from Mojave to Big Sur (OS 11.1), face detection in my Photos library was stuck with 899 (out of about 23,500) photos remaining to be scanned. I performed the configuration reset (item 3 from @jmernin) for each directory listed, one at a time, to see if I could determine which one fixed the problem with photoanlysisd, but it turned out that nothing made a difference. I also performed a malware scan, an SMC reset, a NVRAM reset, and booted into safe mode (which should be the same as removing the login items), but none of these actions helped.

Finally in desperation, I repaired my Photos library (by opening Photos while holding down the Option and Command keys). That resulted in the status changing to zero photos scanned for faces in Photos, but after a while, photoanalysisd started working and almost all of my photos have been scanned (with 281 remaining).

During this process, I have been monitoring photoanalysisd in Activity Monitor, and have observed that it only appears to run for a couple of minutes at a time, and has only run for 13 minutes total over the last 24 hours. Any advice on how to make this program "wake up" or resume scanning faces in Photos, and not quit until all of the pictures have been scanned would be greatly appreciated!

Hello everyone 

Well my scanning started for the first time since Big Sur released  So far 36k of 100k has scanned I have a 2017 iMac 1 TB SSD with 32 gb of RAM. 

Yesterday I installed the newest Big Sur update 11.3, and that started working. 

I had received an email from the Apple media tech that has been contacting on my case over the past two months. He has been really good at interacting with the software engineers on this issue , and had forwarded them diagnostics from my system and such, so as he told me for the last few weeks to be patient because a fix may be coming in an update. Looks like this might be the fix,  

We will see but I am hopeful. He explained “not all the bug notes from the update were posted” and wondered if the update was fixing the issue. 

So to all of you, do the update and see if that works for you. Hopefully it does. The media tech said "If the issue is unresolved after the update, we will want to gather a new macOS Photos Diagnostics and submit them to the engineers”.  

Best Regards all

Don 

@jmernin — Thanks for your hint! I suffered the same problem! (Although) my photos were at the internal SSD at the default location: ~/Pictures/Photos Library.photoslibrary

TL;DR:

1. Remove Login Items -> I did it.
2. Malware Scan -> I did not do it.
3. Configuration Reset -> I didn't nest them at all, but did some grooming. See details below. In particular deleted some orphaned .plist files from Daemons/Agents which were already uninstalled/manually-deleted but their plist file remained.
4. Maybe it was neither #1 or #3 but simply the time factor. Because alltogether the computer was on 4-5 days entirely, mostly idle, photoanalysis alltogether only needed about 1h of accumulated CPU time over those 5days mostly being idle. Had plenty of "missed opportunities". Crappy algorithm.

Feedback to Apple: Going through this on each macOS update is a UX nightmare! Either make that automatic magic behind the scenes work properly (=swiftly, not waiting 4-5 days mostly idle!) or offer an explicit menu command or button "Run curation & face-detection now" with a pause/resume button, should you need your computer resources.

Long Story:

Note that in earlier times my Photos.app Library was an Aperture Library, which was converted when I upgraded from macOS 10.12 Sierra to 10.15 Catalina.

Already when updating to Catalina I suffered from photoanalysisd 's poor implementation. Nothing ever happened. Only when I configured power management to run infinite, once after an overnight session it had kicked in, and still on the morning it was running, and I left it. But then at least it was done.

But this time when upgrading from macOS 10.15 Catalina to macOS 11.1 Big Sur the conversion progress sucked even more!

The start was ok: Upon first launch, the app's view was blocked with an explicit message "Converting…". But after some time it was done. Transparent to the user, obvious progress, no voodoo. Ok, I thought.

After that I imported my last 3 months of photos/videos from two iPhones via USB-cable into Photos.app

Then a subsequent TimeMachine backup. Which was more or less exactly the size of my photo library. So all old master-files and new master-files plus caches/database-files were changed (on a filesystem level), so TimeMachine deemed them backup-worthy.

But then came the long frustrating wait.

Curating showed as "in progress" and face-detection as well.

Left it on for two entire days (48h) and nothing happened, photoanalysisd accumulated no CPU time, the progress indicator in Photos.app UI remained where it was forever.

Had tried to leave my MacBook Pro Retina 15in (Mid 2014) running with screen on, and with screen off.

With Photos.app in foreground or hidden in background or totally off.

With other apps on. With nothing but Finder.

All kind of permutations. To no avail!

The UI sent ambiguous mixed messages:

• Sidebar > Photos > Library: "Leave device connected to power".
  • Quite clear. Whether Photos.app in foreground or hidden in background or totally off not mentioned. So seems to not matter.
• Sidebar > Photos > People: "… Photos will continue scanning your remaining X photos when you're not using the app and your Mac is connected to power."
  • "when you're not using the app" is very ambiguous
    • in foreground but not performing any interaction?
    • in foreground but at another less dynamic screen (not "People", i.e. a static album) "not using" anything such as "editing" or "tagging"?
    • Photos.app was quit, but ran once in the login session, hence "triggered/pushed photoanalysisd to do its work"?
    • freshly restarted Mac, and Photos.app never manually started? photoanalysisd will kick off itself?
  • If the typical Apple magic w/o further ado would work, I would not care for the wording.
  • But as it was not working, no matter what interpretation/permutation I tried, this message drove me wild.

Eventually after adding another 2-3 days after the first days with no noteworthy progress it finally worked.

I opened sidebar > Library > People: And confirmed some faces and additional faces.

Then interestingly you get yet another third kind of instruction: "Updating People… People will finish updating when Photos is in the background."

So yet a third type of instruction! Cannot be more inconsistent and frustrating. Please Apple, ideally get this done in your great beloved "it just works" way (swiftly in the background!) or make it an explicit / blocking action in the UI.

Grooming details (should it help someone)

• I removed cruft files at /Library and ~/Library as instructed by @jmernin after my upgrade to macOS 11.1
• Indented lines are files that I deleted. With a comment on them after the pound symbol.
• See details in this attachment:

According to the Apple Support engineer, because Big Sur comes with a new Major version of Photos, there are some library updates it needs to apply which can take some time. Depending on the size of your library, these updates (and a rescan) could take as long as a week, or more.

Therefore, the nesting changes you've applied above could, in fact, have fixed the underlying issues on your system, and the Big Sur update+rescan changes have only really gotten started properly now. So, perhaps it's still a little early to say on some systems?

Well.........same exact problem, when I updated to Big Sur. I have always used an external. I have 100k total of pics/movies and a 6 TB external Now granted when a new Photo version comes out, in this case version 6, you sometimes have to do a repair/restore with that version within the external and that happens automatically But.........it made no difference, I did a total of three repair/restore, with the same results .......0% of the library scanned. My photos are all there but 0% scanned for People. I can add/import pictures/movies from my IPhone to the external library, but 0% scanned. Having the Photos app running or not running did not make a difference. I decided to first chat with Apple and then spoke on the phone with Apple media tech. He said”you have done everything I would do and added he is seeing complaints like this coming in” His recommendation “ wait for an update patch, that will come in as a software update” . That was a week a week ago the third week of November 2020. So I will be patient and hope it will be forthcoming. I will post an update to this ultimately .

Now for those of you thinking “Maybe my Big Sur app was corrupted when I updated”. Ponder this, I did the Big Sur update on a Mac Mini, attached my external, repaired/restored the external library when it failed, after repeated 0% scan in people, and thought “Yeah maybe it was a corrupted app in Big Sur”.

So, I updated to Big Sur in my 2017 iMac (has a 1 TB SSD.......not a fusion drive ...and 32 GB RAM)......,,,, and same result of 0% scanning. Did a repair/restore twice of the photo library. No change.

So rather than beating this to death and overworking the external , I am going to wait

BTW........I know this is a lengthy response but perhaps seeing what I did may help someone not to repeat it all

Best Regards and waiting

Don

Alternatively to steps 1-3 by @jmernin (which are quite some effort) you could try to Boot into Safe Mode and let your Mac run in this mode without running Photos.app, see whether photoanalysisd makes progress and check visually in Photos.app here and then. And when done, boot normal again. Then all your startup items, daemons and agents are re-enabled. No need for manually doing this.

I did not test this, but came up with the idea afterwards.

For a HowTo which is proven to have worked, check:

• @jermin Solution is to remove potentially orphaned/conflicting background daemons/agents/launchitems
• @putzfetzenORG: An even more detailed report, possibly not all steps necessary as outlined in above post

Hello jmernin,

I think it is a good idea to share the folders that actually got nested to potentially help others narrow down things and change less of their system. I'd like to point out that I really do think that Step #3 of your list is what did it for me.

My system is brand new, a Mac Mini M1 with OS installed from factory. I don't have anything installed (it is not my main system) and Photos was the first thing I was trying to get to work. My photo library is the only thing that I am using from the old system, and this is through an external hard disk (it always has been there) that I moved to the new system. Because of the fact that I was having the same issue with a 'minimal' system, I think this is the best example of what really needs to be done to fix the problem. And, unlike some other comments, I don't think with my case it was a 'coincidence' that it worked. I had spent several weeks doing different scenarios to make Photos to start scanning without trying to 'fix' anything. This was the first time I tried a 'fix' and the library started scanning immediately after that. The scanning of all my ~90k photos as finished by the way. :)

So, I went through and checked the folders that still had the 'New Folder With Items' that I created during Step 3# of your instructions. I'll list them below. But first, I'd like to point out that after more digging, it seems that I was mistaken once again as to the 'Libraries' available in MacOS. It turns out there seems to be (at least) 3x locations... There's "~/Library", "/Library" and "/System/Library", and all have different contents (at first I thought some were mapping to the other but that doesn't really seem to be the case. I did not make any changes to "~/Library" because that wasn't visible through Finder (although you have since mentioned how to get there through key combinations). Also, several of the folders were empty (again, new system) or I wasn't able to nest any of the items in them (I'm guessing permission issues).

The only folders that I apparently nested are the following:

• /Library/Caches
• /System/Library/Caches

I was going to itemize the files within those folders that were nested, but being 'Caches' I figure it doesn't really matter, you should be able to just nest all without concern as long as all Apps are closed.

So, considering that I don't think you actually did any nesting in '/System/Library', I would say that if someone is interested in trying to find a 'minimal' approach, I would suggest starting by trying to nest '/Library/Caches' first... Then if that doesn't work trying other folders from your list. Also to remember, as was mentioned here before, the Photos app has to be left closed. I just restarted the system and didn't open anything and let if fall asleep for the night.

Hope this helps others! If someone tries this approach of only nesting '/Library/Caches', please let us know the outcome...

Good luck!

Are there any Release Notes with the newer version that could indicate what has changed in the newer version of the Photos app?

Also, if you try creating a new library and importing a couple of photos to that, do the faces in those photos get scanned?

Thanks for the work you're doing to try to debug this.

putzfetzenORG wrote:

I opened sidebar > Library > People: And confirmed some faces and additional faces.

Then interestingly you get yet another third kind of instruction: "Updating People… People will finish updating when Photos is in the background."

I had the issue where photos wasn't scanning and 'fixed it' and then for months it has been as you described in the quote above "Updating People...". Did yours ever finish updating people? I have a lot of people that it detected but there seems to have been no additions for months. Yet, if I go through the pictures, they all have 'circles' on peoples faces but are not recognized... The latest MacOS update does not seem to solve this...