Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

Photo Library - iCloud duplicates how to remove

All, anyone know anything about OSX/IOS "Photos" app ability to detect and remove duplicate photo9? My "Photos" library has 63,299 photos & 2135 videos as of May 2016 and at least 10,000 + of these photos are duplicates (based on a scan by Photosweeper). I have checked the result of the Photosweeper scan by doing a manual visual comparison and check of original image files in iphoto library (using show location function) and confirmed these 10,000+ photos are identical duplicate images with different file names located in different parts of photo library database.


So what is going on and how do I stop 10-15% of my image library being duplicate files!!! I will raise this directly with apple support as paying for the 1TB icloud service to cover my very large "Photos" library which should probably just be a Large Photo Library!!!!


Some other comments for those who have a duplicate problem:

- The old iPhoto application used to "detect duplicates" when importing but the new "Photos" app does not do this. Anyone know about this?

- I saw some documentation from apple saying that the icloud scans for duplicates in "Photos" when in the icloud but clearly not working or it does not work if the duplicates also exist in the imac version of the "Photos" library.

- Watch out for apps that claim to find duplicate "photos" and are recommended on various websites, a number of these only work with the old iphotos app not the new "Photos" app (as I found out after paying and downloading them). If you need a photos duplicate remover that work with "Photos" check review and stay away from any app that does not clearly and expressly state that it works with "photos" and has been reviewed since the release of the Photos app.

+ I paid US$10 for "Gemini" the duplicate finder and was a complete waste of time as did not work with "Photos" app (this was not clear from documentation or support and when I raises this with Macpaw support they told me gemini does not support photos and told me to buy another macpaw app - I told them to go jump in a lake).

+ I found Photosweeper very good as you can set for exact match or variables (eg accidental burst photos that are 0.1s apart but 99.99% identical) and you can then right click to view original file / image in library to manually check for duplicate (if you are paranoid like me) - there are lots of other paid apps that also do this so shop around

Posted on May 27, 2016 5:31 PM

Reply
Question marked as Best reply

Posted on Nov 2, 2017 10:13 PM

All, I spoke to Apple support today (3 Nov 2017) and they confirmed that as of today Apple Photos App has no way to identify or find duplicate photos in your photo library or icloud library.

If you would like to change this please log a "Photos Feature Request" or "Photos Enhancement Request" using Apple "Photos" app feedback form below to help resolve this problem


Apple support "photos" specialist confirmed following 3 Nov 2017:

1) The new Photos app and old iphoto app had a way to search for duplicates on import this does not work once the duplicate photos are imported into library (by error or otherwise)

2) Apple photos has no way to search for duplicate photos, they suggested that users "manually search for duplicate photos".

3) Apple support do not recommend third party apps to remove photos as they can damage the library.

4) They acknowledged this approach is less than ideal when you have 20,000+ photos (or in my case 80,000 + photos) and suggested that all users who have concerns should log a new feature request using apple feedback form (URL below for IOS and OS Mac Photos feedback form).


Please log a "Photos Feature Request" or "Photos Enhancement Request" using Apple "Photos" app feedback form below to help resolve this problem


Submit Enhancement Request using Apple IOS "Photos" app feedback form

https://www.apple.com/feedback/photos-ios.html


Submit Enhancement Request using Apple Mac OS "Photos" app feedback form

https://www.apple.com/feedback/photos.html


Cheers MR HK

22 replies
Question marked as Best reply

Nov 2, 2017 10:13 PM in response to MrHk

All, I spoke to Apple support today (3 Nov 2017) and they confirmed that as of today Apple Photos App has no way to identify or find duplicate photos in your photo library or icloud library.

If you would like to change this please log a "Photos Feature Request" or "Photos Enhancement Request" using Apple "Photos" app feedback form below to help resolve this problem


Apple support "photos" specialist confirmed following 3 Nov 2017:

1) The new Photos app and old iphoto app had a way to search for duplicates on import this does not work once the duplicate photos are imported into library (by error or otherwise)

2) Apple photos has no way to search for duplicate photos, they suggested that users "manually search for duplicate photos".

3) Apple support do not recommend third party apps to remove photos as they can damage the library.

4) They acknowledged this approach is less than ideal when you have 20,000+ photos (or in my case 80,000 + photos) and suggested that all users who have concerns should log a new feature request using apple feedback form (URL below for IOS and OS Mac Photos feedback form).


Please log a "Photos Feature Request" or "Photos Enhancement Request" using Apple "Photos" app feedback form below to help resolve this problem


Submit Enhancement Request using Apple IOS "Photos" app feedback form

https://www.apple.com/feedback/photos-ios.html


Submit Enhancement Request using Apple Mac OS "Photos" app feedback form

https://www.apple.com/feedback/photos.html


Cheers MR HK

Jan 29, 2017 7:34 PM in response to ca_boarder

Guys this app was brilliant. It literally took 3 minutes to remove all my duplicate photos. Have have over 17000 photos and had about 1000 duplicates.


Just to give you an idea of how easy it is to use.


1) Download it for free from App Store (there's a pro version but you don't need it)

2) Point to your Photos library

3) It scanned 17000 photos in about 6 seconds

4) It discovered 1000 duplicates in 5 seconds

5) It moved the duplicates to a special Album in Photos called duplicates

6) I deleted all the photos from that Album

7) I deleted all the photos from Recently Deleted

8) I deleted the album

9) I removed the app

10) I gave it a 5 star rating


Literally a couple of minutes and you're done.


Great suggestion mate, was pleasantly surprised.


Here is the link again in case it looks out of context

https://itunes.apple.com/us/app/photos-duplicate-cleaner/id592704001?mt=12

Jan 31, 2017 9:04 PM in response to MrHk

If you are using a third-party app to detect duplicates for you, why don't you delete them once they have been found?


Photos does check for duplicates when you are importing images from another device, and it gives you the option to Import All or Import Only New.


Gemini from MacPaw is a utility app to check for duplicate files on your system, not in specific apps.


And from reading your post, it seems you have already solved your issue, you're just not deleting the duplicates when you find them? If that's the case. Just delete them.

Jan 31, 2017 9:27 PM in response to Nick Jeremiah

Gemini from MacPaw is a utility app to check for duplicate files on your system, not in specific apps.

Gemini and MacPaw's CleanMyMac are Malwear and often damage or destroy Photos adn iPhoto libraries - neither should ever be used wit Photos and IMHO should never be used at all - they are a total waste of money and may actually damage things



LN

Feb 25, 2017 6:10 PM in response to MrHk

Yes be very careful of duplicate elimination programs - many (including CleanMyMac and Gemini do not work correctly the Photos (or iPHoto fro tha matter) and may damage or destroy your library


As to safe duplicate programs PowerPhotos, Photo Sweeper for Photos and Duplicate Annihilator for Photos are safe


and Photos does check for duplicates when you import photos and it does not upload duplicates to iCloudPhotoLibrary - but you do not have duplicates, you have different photos with different names not duplicates which are identical

identical duplicate images with different file names

Those are similar photos not duplicates - PhotoSweeper does the best at finding them


and back to your question the way to remove them is to select them and press command delete to move them to the recently deleted album where they will be available for 30 days or until you delete them



LN

Nov 13, 2017 2:37 AM in response to MrHk

HK


Thanks for the update re your discussion with Apple Support - I have followed your advice and sent in a request as well


In the meantime can anyone confirm if there is s trusted way to sort this problem out using 3rd party software or other techniques


If the explanation on what to do could be "idiot proof" then that would be appreciated as I am not that experienced with using the features on my imac


thanks Sandylodge

Nov 17, 2017 1:13 AM in response to warrior200

Hi Warrior200,


I’ve had this problem since iPhone 4S, iPhone6 Plus, iPhone 7 Plus and now iPhone X. I have been dealing with thousands of duplicates. I’m talking 20 - 30 duplicates of a single photo that I only had one to start. I’ve turned off iCloud Photos so many times, on all devices. Cleaned up every duplicate on my MacBook Pro (main Local Library), taken backups of the Photos Library. As I now have a completely duplicate free library that has been backed up, I made sure iCloud was empty. I then turned iCloud Photos back on, on my MacBook Pro to let them upload a clean library. It ran overnight. Everything seemed alright. I turned iCloud Photos on for both my iPhone 6 Plus and my iPad Mini to let them download the clean library off the Cloud. Low and behold, there were even more duplicates created than before. It seem like the more I try to clean it up, that more get created on the next sync. It’s pain staking and no matter how good a duplicate remover is, it’s no fun dealing with thousands of photos over and over, when it’s just going to happen again and again and again.


I’ve even done this without signing out of the cloud, that once deleted, your asked if you want to remove this from all devices.... saying yes works, but they return. It’s like every device has its own naming convention for the files and the cloud uploads it from one device and distributes it to all other devices, and keeps cycling. 4 devices, 4+3+2+1 = 10 copies for each unique name, even if its the exact same photo.


What a pain in my Arse!!!


Apple will have to fix this at some point. Trillions of duplicates with Petabytes of wasted drive space is an un-necessary cost to the consumers.


I understand Apples stand point, to run some sort of Mass Update to clean up the iCloud servers could potentially cause permanent loss of peoples photos if the person relies on the cloud for storage. Foolish on the users part. Get a network drive and back your photos up with RAID 10. Pretty safe bet that even if Apple were to accidentally remove photos, you’ll have them safe and sound on your own storage.


Good Luck!


I hope Apple reads these comments and takes them into consideration for fixing things!


Sincerely,

Daniel

Dec 30, 2017 9:30 AM in response to MrHk

This just happened to me. I upgraded from a 7Plus to the new X, and went from 15G in my photos to 168G overnight! This happened from restoring from iCloud backup!!!! I had I upgrade my phone because my 7 was slow and now I’m running out iof space in my iCloud! I talked to support and they confirmed “something happened”, but there iis nothing they can do to help me!


I will be moving to Google Drive as soon as I get this corrected. The good thing is I can share a data plan with my family instead and us all paying for a 200G plan for each phone!! Started Mgr know regret not going to the google phone now...

May 27, 2016 5:39 PM in response to MrHk

Photos does detect duplicates when importing photos and wehn syncing with ICPL


Photos does not scan for duplicates but does check for them during import and during upload to IPCL


There are a few duplicate programs that are tested as safe with Photos including PowerPhotos, PhotoSweeper for Photos and Duplicate Annihilator for Photos - do not use one not tested and documented as safe\


And I'm not clear what your post is about since you ask how to find duplicates and then provide a good answer - photoSweeper - that is one of the safe, effective ways to remove duplicates for Photos


LN

May 27, 2016 7:56 PM in response to LarryHN

Larry, I thought my question was pretty clear, I am asking "how do I stop 10-15% of my image library being dulicatae files", the third party duplicate programs can help you fix the problem manually by doing scans after the fact but not stop the problem . I am asking for a prevention and why this is occurring

Photos does not detect duplicates when importing photos (not sure what syncing with ICPL is) - I have tested this numerous times.

- The old iphoto used to idnentify duplicate when images were being import and ask if you wanted to keep duplciate replace it or skip it

- The new Photos allows duplicates to be imported with no warning or ability to replace etc

Nov 8, 2016 4:35 PM in response to MrHk

I have the same problem with hundreds if not thousand of photos duplicating.


I have completely deleted all images off my devices and uploaded them again after running a duplicate finder separately on my pc.


After a few days I ran photosweeper on my MacBook and it found around eight hundred duplicates that definately were not uploaded!


Would love to know why this is happening.

Nov 8, 2016 9:23 PM in response to MrHk

There is one thing that I can think of that would probably work, although it is rather inconvenient.


You could try exporting all your library to a folder on your Mac, (make sure you select 'export original' otherwise you will find some of the images can lose quality.) Be aware that this will delete any adjustments such as cropping you have made in photos.


Once they are exported and you have checked they are all there, delete all the photos in the library (go to the 'all photos' view > click on the first image displayed > scroll down to the last image while holding shift and then click on it. That will select all images). Oce you have them all selected, delete them (alt option click > delete), they will then be moved to your recently deleted folder in iCloud photos, where you should keep them for the first part in case anything goes wrong.


Once you have done all of that, try reimporting all the photos you exported. After it has processed some of the photos, it will ask you if you want to import duplicates. Select no and then it should only import one copy of each!


I hope that helps!

Nov 9, 2016 1:25 AM in response to warrior200

You may be surprised, if, as I can see from your account, you are exporting them on a retina Mac book Pro, they may be a lot quicker than you expect! The importing and duplicate detection is the thing that takes a while. If your library is about 20GB in size, I would expect that to take about half an hour, though it may be slower if you are using an older model. Good luck!

Photo Library - iCloud duplicates how to remove

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple ID.