Can someone explain how Photos stores its files and how to avoid/delete duplicates?

Goal: get all my full size photos and videos in one location so I can view in Photos and recover hard disk space


Secondary goal: would be willing to achieve the above through other photo organizing software


Questions about storing and organizing pictures and video in Photos:

1. Which photo library files must I keep on my computer to view photos in Photos?

2. Can I remove the iPhotos migrated library file and recover that hard drive space?

3. Can I remove the Masters folder and recover that space?

4. How can I ensure photos saved to hard drive prior to installing Photos gets added to the Photos Library, and are they physically saved in the old and/or new new location?

5. Should I check the Metadata box?


Hardware:

mid-2012 MacBook Pro, 2.9GHz Intel Core i7, 16GB RAM, 1TB SSD, El Capitan 10.11.6, Photos 1.5

iPhone 6S with iOS 11.3 is primary camera now

Occasionally download from an SD card from a digital SLR and a HD camcorder


Software:

Duplicate Detective 1.99

PhotoSweeper 3.2.1.

Photos 1.5 settings:

Library location is Photos.Photoslibrary

Summarize Photos is checked

Copy items to Photos Library is checked

Metadata is NOT checked (Include location information...)

No longer can use iPhotos (no 9.6.1 version)


My Masters folder is 447 GB

Photos.PhotosLibrary is 451 GB

iPhotos Library.migratedphotolibrary is 232 GB (merged in 2014)

Hard disk holds 1050 GB, but currently shows 980 GB used.

Obviously, there are data files being double counted in these numbers because my SSD holds only 1050 GB.


My Masters folders has folders for every years since 2014. I used iPhotos since 2013, and I've downloaded digital pictures from various non-Apple cameras since around 2005.


I've run PhotoSweeper and Duplicate Detective several times and deleted the photos from Photos and emptied the trash to get to the above data file sizes. I don't know if all my files in the Masters folder are also in the Photos library, but I thought that's how the system worked. I recently had multiple additional folders full of pictures (full size, not just thumbnails) that I moved to an external hard drive, but then I noticed some of the pictures in Photos became thumbnails and would not open to full size.


Any suggestions or general knowledge would be appreciates.

MacBook Pro (13-inch Mid 2012), OS X El Capitan (10.11.6)

Posted on Jun 30, 2018 10:16 AM

Reply
Question marked as Top-ranking reply

Posted on Jul 1, 2018 1:52 PM

1 - all of them - there is no reason to ever go inside the Photos library and there are no user serviceable parts in it and nothing can be removed or modified using the finder or any software other than Photos


2 - yes if you have not used it for 3 months or more - just archive it and trash - but you will recover much less space than you expect due to the use of hard links so nothing is duplicated between the two libraries


3 - no - again you must NEVER make any changes to the content or structure of the Photos library - it is best to never even go into it for any reason


4 - import them to Photos - either drag to the Photos icon in the Dock or use the Photos file menu ==> import command


5 - yes for any metadata you want to view


Secondary goal: would be willing to achieve the above through other photo organizing software

That is not possible except by duplicating the photos and using another photo organizer on the copies - Photos will use external Photo editors but you can not use two different organizers on the same photos


I now nothing about Duplicate Detective but it is directly deletes anything from the Photos library you have destroyed your library using it - you can only use duplicate elimination programs that identify photos so you can then use the photos application to delete them - PhotoSweeper does do that and is safe


--Again always remember

you must NEVER make any changes to the content or structure of the Photos library - it is best to never even go into it for any reason


Photos uses a SQLite database and its library is not intended for direct use by any other program - everything must always be done using the Photos Application


LN

Similar questions

5 replies
Question marked as Top-ranking reply

Jul 1, 2018 1:52 PM in response to PigeonF16

1 - all of them - there is no reason to ever go inside the Photos library and there are no user serviceable parts in it and nothing can be removed or modified using the finder or any software other than Photos


2 - yes if you have not used it for 3 months or more - just archive it and trash - but you will recover much less space than you expect due to the use of hard links so nothing is duplicated between the two libraries


3 - no - again you must NEVER make any changes to the content or structure of the Photos library - it is best to never even go into it for any reason


4 - import them to Photos - either drag to the Photos icon in the Dock or use the Photos file menu ==> import command


5 - yes for any metadata you want to view


Secondary goal: would be willing to achieve the above through other photo organizing software

That is not possible except by duplicating the photos and using another photo organizer on the copies - Photos will use external Photo editors but you can not use two different organizers on the same photos


I now nothing about Duplicate Detective but it is directly deletes anything from the Photos library you have destroyed your library using it - you can only use duplicate elimination programs that identify photos so you can then use the photos application to delete them - PhotoSweeper does do that and is safe


--Again always remember

you must NEVER make any changes to the content or structure of the Photos library - it is best to never even go into it for any reason


Photos uses a SQLite database and its library is not intended for direct use by any other program - everything must always be done using the Photos Application


LN

Jul 1, 2018 7:47 PM in response to LarryHN

Thanks for the reply.


I've done most of what you suggested. I moved the iPhotos migrated library file to an external hard drive for safe keeping and trashed and emptied it off my hard drive. It saved a lot, but not the full displayed amount. I hadn't used anything form iPhotos in years.


Duplicate Detective is just a third party software that compares files to find duplicates. It doesn't work inside the Photos library like PhotoSweeper does. I used it on my Master folder, and it found 120GB of duplicate files from when I must have imported pictures and video a second time. I trashed and emptied those files, and Photos seems fine. I kept the oldest files, which were probably the ones Photos was using. Maybe I got lucky?


Previously, I had run Photosweeper on Photos and removed all the duplicates it found inside of Photos. I followed the instructions and let PhotoSweeper moved the duplicate pictures to a temporary folder within Photos and then followed its instructions to delete them. Trash emptied. That got my Photos folder a little smaller.


I had also imported all the remaining pictures outside Photos into Photos. I would run PhotoSweeper after each import to see if those pictures had already been imported. Eventually, I got all my pictures inside Photos, but it seems I am still missing some.


My Photos library is 451 GB. I know some of it is soccer matches on HD that I just need to remove. Each match was too big to fit on a DVD...bigger than 4.7GB. I probably have 100 GB of soccer video that I don't need.


Except for deleting duplicates in the Masters folder, I've done what you suggested and it seems to have worked.


Thanks for the info.

Jul 1, 2018 8:04 PM in response to PigeonF16

Duplicate Detective is just a third party software that compares files to find duplicates. It doesn't work inside the Photos library like PhotoSweeper does. I used it on my Master folder, and it found 120GB of duplicate files from when I must have imported pictures and video a second time. I trashed and emptied those files, and Photos seems fine. I kept the oldest files, which were probably the ones Photos was using. Maybe I got lucky?

This makes no sense since the masters folder is inside the photos library -- and if the program or you deleted any files directly from the library then you have a corrupted library and someday it will bite you - you simply can not do that without causing problems


good luck but if your photos have any value to you quit messing around inside the Photos library


LN

Jul 7, 2018 10:03 PM in response to LarryHN

A few years ago, someone posted how to find the Masters folder where iPhotos/Photos keeps all your pictures and video. I pinned it to the side dock. Having been a PC guy for 30 years before getting a Mac, I wanted to be able to find my pictures. All I ever did was copy files from the Masters folders to load onto an external HDD. I started doing that to move large video files for the school's soccer coach. I couldn't get the video to load from Photos for whatever reason, and I found that simply copying the file directly was easier.


You're probably cringing right now, but it's not as bad as it sounds. I have Time Machine backups on multiple disks. I don't know how PhotoSweeper works, but it allows you to move all the duplicate pictures to a folder, and then you manually delete them from Photos. PhotoSweeper works great. On the other hand, using Duplicate Detective on the Masters folder allows you to find duplicates without using Photos. In fact, I wouldn't open Photos when I did it. I haven't noticed any problems with Photos after running Duplicate Detective on the Masters folder. I haven't had any library issues. What I don't understand is how PhotoSweeper missed 120GB of duplicates that Duplicate Detective found in the Masters folder. I went back into Photos and found the same duplicates that DD found, and this was after having run PhotoSweeper several times and finding no more duplicates.


Now that I've used DD, I shouldn't need to do it again. I think I just got lucky somehow. I understand what you're saying about not messing with the Masters folder. I considered going to Google Photos, but I don't like the idea of Google scanning my pictures for faces and marketing. That's why I have to learn what's best for using Photos.


Thanks for your advice (now that I'm following it).

This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

Can someone explain how Photos stores its files and how to avoid/delete duplicates?

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.