Corrupted Originals, the Insidious Danger - How to Protect our Photo Libraries
Sooner or later you may encounter this problem: We open an image in our photo library for editing or to view enlarged, that we imported years ago and have not looked at for a long time. And then we notice to our great horror, that large blocks of the image are missing and replaced by gray or black blocks or other parts of the image (see the examples below). When we try to restore the image from a Time Machine backup, the corruption may appear in all backups, so it has been happened a long time ago, but we did not notice it. This kind of damage is typical for JPEGs. It looks like Photos, Aperture, or iPhoto had damaged our Photos, but the photo applications are not the culprit in this case. JPEG corruption can occur, when JPEGs are stored on disk blocks, that got damaged, because the drive is old, or when we are storing image files on external storage devices like SD cards and USB flash drives, or drives that are shared on a network. Flash drives need to be frequently connected to power to preserve the data, and network shares may be causing transmission errors or synchronisation errors. Because JPEGs are in a highly compressed format, a few flipped bits will suffice to make large parts of the image unreadable. Whenever media files are copied or moved (during a backup, moving to a new computer or volume, download by USB, from iCloud, restoring from a backup), there may be transmission errors, and drives will fail some day. If we discover the damage later, it will be hard to say what has caused the corruption.
The problem: Why are the damaged originals hard to spot
Our photo libraries (Photos, iPhoto, Aperture) are keeping the original image files paired with the current versions of the photos. The originals are safely tucked away in the library, and in the user interface we are working with working copies of the current versions - thumbnails and previews. Because we are usually seeing the previews and thumbnails, and only seeing the originals, when we want to edit a photo or view it enlarged, it may take many years, before we notice, that an original image file has become corrupted. The originals may be slowly dying in the background, and we will not be aware of it, until it is too late and the last good copy has been purged from our Time Machine history. The previews and thumbnails we are seeing in the user interface are just a delusion of a perfect world, while the originals are already flawed.
Our Time Machine backup may not suffice to restore the originals; even if we have a very large backup drive, but Time Machine cannot keep an archive with a full history. Older backups are purged, when Time Machine runs out of storage. Last year I had to go back in time to the year 2002 to recover a corrupted JPEG from an old iPhoto Library from 2002.
How can we protect our Photo Libraries?
I am using these four precautions to be able to recover from a JPEG corruption:
- Checking the images when we import them: Directly after importing the images we should view each photo enlarged, otherwise we may only be seeing the embedded JPEG thumbnail and not the image itself and not notice transmission errors.
- Backing up the originals: Once we are sure, that the originals have been imported correctly, without transmission errors, we should make a backup copy on a different drive (and check this copy as well). Only then we can delete the originals from the camera or card. Do not use the automatic "delete after Import" when importing photos, or you may be deleting your photos, before you could check them and back them up. Many irreplaceable photos have been lost by "delete after Import". The additional backup of the originals will come in handy, if the Photos Library should become damaged beyond repair or you want to downgrade to an older system version. Or if you are using Photos 5.0 on Catalina; it is not possible to recover the originals with all file attributes preserved from a Catalina Photos Library, because Catalina is renaming the files internally (How Photos 5.0 on Catalina Manages origin… - Apple Community)
- Early diagnosis - a regular check of the originals: Because we can only see our original image files by deliberately looking for them, I am regularly viewing my important images in the Favorites folder enlarged in Edit mode, so I can be sure all is well with them, before they will get purged from the Time Machine backup. There are also third party applications to check JPEG files for corruption. I have found Corrupt JPEG Checker very useful, to scan the Masters (originals) folder in my Photos Libraries.
- Hardware test: Ensure that the memory and the drives are working correctly. See: How to use Apple Diagnostics on your Mac – Apple Support. And I would not keep the external volumes lying around without regularly using them.
There are some programs, that claim to be able to repair damaged JPEGs, but I have not tested any of them, so I cannot recommend some of them.
Anyway, backing up the original image files is safer than relying on a repair tool.
Here are some samples of damaged JPEGs, that I found in my library, at that time still on Aperture. Later I found similar damaged files in a Photos Library.
https://discussions.apple.com/thread/3620471?answerId=17173902022#17173902022