macOS Catalina photos - remove duplicates?

PhotoSweeper - finds all duplicate photos regardless whether they are stored as the original photo or displayed in “Albums” . Its “AutoMark” feature marks photos in albums or the actual original photo for deletion. BUT if you let it delete the original photo, then all other occurrences of the photo that might be in albums will also be gone. (The album will have a pointer left that points to a deleted original and therefore can’t be displayed) 

Therefore I’m dismissing PhotoSweeper as a workable solution.

That's how it's supposed to work. PhotoSweeper finds duplicate files in a library, not duplicate or separate occurrences of an image like in multiple albums. No app can do that.

The following is how photos are deleted in a library. Pay particular attention to Note 2 and 3.

Deleting Photos from a Photos Library

1 - regardless of where you are in the library, i.e. Photos, Memories, Favorites, People, Recents, Imports, albums, smart albums or projects, select the photo(s) you want to delete and use the key combination of Command+Delete to move the photos to the Recently Deleted smart album.

2 - click on the Recently Deleted smart albums in the sidebar.

NOTE 1: If you're using Photos 1.2 or earlier use the File  ➙ Show Recently Deleted menu option.

3 - in the Recently Deleted album you can opt to let the photo automatically delete after the 30 day waiting period is up, delete them immediately or restore them to the library.

NOTE 2: deleting a photo from an album, slideshow, book, etc., with the Delete key only deletes the photo from that item. There is no key combination that will delete a photo from a project and move it directly to the Recently Deleted smart album.

NOTE 3: deleting a photo from a Moment or Collection (the Photos window) or the All Photos window deletes ALL occurences of that photo in the library.

PhotoSweeper and PowerPhotos are the only two apps I recommend for removing duplicated image files from a Photos library.

That is a slightly misleading line in the Catalina preview. The Days View is just hiding the duplicates - simply sweeping the clutter under the carpet now, for easier browsing, but not removing the clutter and duplicates.

You still have to delete the duplicates manually, which is now even harder, because we cannot see them in the days view and we cannot see the titles and filenames in the All Photos view. The useful Moments view is gone now. I am recreating the Moments by creating albums for each shooting session. I have to use albums and not smart albums, because the smart albums can no longer be sorted manually. And the smart albums do not sync to my iPhone and iPad.

I have been using Photos since it replaced its predecessor and I use it on MacBook Pro 2011 Unibody (High Sierra) with SSD and on a MBP 2013 Retina (Catalina) and personally I have been happy to rely upon Power Photos for Duplicate cleaning etc. Try it free first if you like at : https://www.fatcatsoftware.com/powerphotos/

Helpful support too if in difficulty... like with MacOS upgrades ;-)

I just wish I could find a similar app for duplicate files of other kinds!

Colin

I fear that in choosing to mass all your photos in one cloud file you have created the one big problem that you say you prefer.

If you seek a means of searching that file for duplicates my suggestion would be that you ask support@fatcatsoftware the authors of Power Photos for their advice. I chose to keep my several photos files separate off the cloud but on several Macs and Power photos have been a rock I have been happy to be able to look to for advice. I commend you to ask them too.

sorry I cannot help but happy to be able to commend you to another who I believe to be most likely to be able to help.

good luck

Colin

You have more faith in iCloud Photos than I do, Lethal_777. I always try to make sure that at least one of my Macs has Download Originals set, so that I can take backups of my photo library that include full res photos. As a side benefit, I can run PowerPhotos on that Mac without running into the issue you cite.

What are you referring to by "IT"? Read this user tip and then restate your problem and how you got to where you are now in detail: Writing an effective Apple Support Communities Question - Apple Community

Have you spoken to FatCat Software?

no one knows it better,

c

I think it was more a poorly described feature and not a feature advertised and not included.

I’ve been struggling with finding the best method to delete duplicate photos within my “Photos Library”

Photos allows for creating “Albums” that are made up of photos stored in the top level “Photos Folder”. Copies of these photos that are put in “Albums” are actually not photos but instead pointers back to the source photos. The “Photos” app stores the original photos in the “originals” folder and utilizes a sqlite DB to manage pointers to photos inside “Albums”

I’ve tried using the “PhotoSweeper” and “Photos Duplicate Cleaner” apps, but neither one removes duplicates correctly.

PhotoSweeper - finds all duplicate photos regardless whether they are stored as the original photo or displayed in “Albums” . Its “AutoMark” feature marks photos in albums or the actual original photo for deletion. BUT if you let it delete the original photo, then all other occurrences of the photo that might be in albums will also be gone. (The album will have a pointer left that points to a deleted original and therefore can’t be displayed) 

Therefore I’m dismissing PhotoSweeper as a workable solution.

Photos Duplicate Cleaner - seems to work smarter, it only looks for true duplicates inside the “originals” folder of the library, ignoring duplicates (pointers in the albums) It also AutoMarks duplicates from its discovery. Sofar so good, BUT lets say it marks photo #2 (the duplicate) for removal and a photo inside an “Album” points to photo #2 (the duplicate) then the photo in the “Album” will also be gone.

What should actually be happening is that if a duplicate is marked for deletion and includes pointers to the photo in one or more albums, a “symbolic link” or pointer from the marked photo to be deleted to the remaining original photo should be created before deleting the duplicate in order to preserve the “copies of the photo” that occur in “Albums” or all its pointers within albums should be remapped to the remaining original photo.

Now that I hopefully explained the problem, can you recommend a better duplicate remover that solves the issues outlined above.

BTW - Neither of the two apps explain to a user that most likely they will loose tons of their photos (either originals or inside albums) by using their apps. In my case it identified ~5,000 duplicates for removal and I fret over deleting them with the apps mentioned.

Thank you for sharing your knowledge and experiences. I purchased Photos Duplicate Cleaner and it is helpful. However, I fear losing photos in the albums. You hit the nail on the head about putting the pointer in the albums to the "saved original" when the duplicate is deleted.

Coming from Windows, I find it extremely frustrating that Apple doesn't have a feature to identify which albums a photo is in. As sophisticated as Apple products are, I see this as a basic design flaw! Apple comes up with fancy features yet misses the basics...for example a "green eye" removal for pets, which is just like red eye but for pets! There are millions of pets and this is a big hole.

You can use this AppleScript to find out what albums a photo is in: How to find out the albums a photo is in: Jacques Rioux's Script, version 2019

The script compiled as an application can be downloaded from this tutorial site:  P01 - Applescripts from Photos’ User Tips Compiled as Applications