Apple Photos says referenced files are "missing" after moving external drive to another machine

Question

Level 1

9 points

Apple Photos says referenced files are "missing" after moving external drive to another machine

Mojave 10.14.6

Photos 4.0

I have my Photos Library on an external drive, and on the same drive are the photo files in folders as referenced files. All has been good.

I physically disconnected the external drive and connected it to a different Mac, running same version of Mojave and Photos. The referenced files all come up "missing".

Drive names were not changed. File paths were not changed.

Repairing the Library did not correct the problem.

The external drive is formatted as Mac OS Extended Journaled.

Ignore ownership on this volume is checked.

I have run first aid on the drive and it is fine.

In Photos, when I use the option to find the missing file, it always takes me to the correct folder for the missing image, I only have to hit return to accept it, and it works, the file is no longer missing. However, with 94,000 files, it would take a tremendous effort to fix them all.

As a test, I made a new Photos database on a different (similarly prepared) external drive and handful of images. As before, when I physically moved the drive to the other Mac, the same problem occurred. All of the files were "missing". If I move the disk back to the first computer without finding/fixing the missing files, it is fine (photos are not "missing").

To further the experiment, on the second computer, I reimported the handful of test photos (from their same referenced locations). They were not recognized as duplicates on import, even though they are the exact same files in the exact same locations. They imported successfully, and there are two sets of the same images, from exact same files on disk in Photos. The newly imported files are not "missing" according to Photos, but the files imported on the first computer continue to be "missing". So, I moved the external drive back to the first computer, and, the files imported first, from that computer, are no longer "missing", and the files imported on the second computer are "missing". In other words, the files are not "missing" as long as the external drive is connected do the computer where they were originally imported.

I spent an hour on the phone with Apple Support and they were not able to resolve the problem, though they tried. This was before I replicated the problem with a fresh library, for what it's worth.

As far as I can tell, the drive and the files are fine, but for some reason the Photos app cannot properly find the files when I move the external drive to a different machine.

I have also tried moving the external drive to a third computer running Big Sur and Photos 6.0. Same problem. One difference is that Photos 6.0 on Big Sur is revamped, and so it does an migration/update process on the Library. Outcome is the same.

I have backups of the files and the library. No worries in that regard.

My goal, ultimately, is to successfully migrate to Big Sur and Photos 6.0.

I have not attempted migration through Time Machine or Migration Assistant. I am not optimistic about those routes based on what I've read online. I do not have room on the system disks of any of my machines to put the photos there. I suppose I should test these options.

I have opened the Photos.db file and investigated RKVolume and RKMaster tables. Everything seems in order, but I could easily be missing something. I've compared before and after moving to different computer.

I have explored the possibility that XATTRS are playing a role. It seems they are not, but again, I could be missing something.

The fact I can replicate the problem with a fresh Photos library and as few as 1 file suggests to me that my Library is not corrupt. And the fact I have replicated it on a different external disk suggests to me that the disk is fine. I've tested with a combination older and brand new photos with the same results.

I would rather not consolidate my files within the Library, but I might resort to that, if tests show that it resolves the problem.

I'm tempted to say there is a serious bug in Photos. I know others have had similar sorts of issues, though more often having to do with renaming a volume. I keep hoping that I'll find some little trick that makes it all work out. My gut tells me, at this point, that the problem is not so much with Photos as it is with how the Mac manages external drives.

Every several years I consider moving to something else. I tried and was happy with Aperture. From there I went to LightRoom. I didn't like it. I came back to Photos. It is hugely problematic that there is no way to export everything and reimport it without loss of effort invested in organizing and editing. I am considering moving to file system approach using GraphicConverter and ACDSee PhotoStudio for Mac, but this would be a major change.

Anyone else having the same problem?

If anyone has any ideas about what is causing the files to be "missing" in the way they are, I'd appreciate hearing them. Thanks.

Mac mini 2018 or later

Posted on May 6, 2021 7:34 PM

Reply

Answer 1

Best reply

Old Toad

Level 10

207,991 points

May 7, 2021 10:49 AM in response to jweise

If you have a referenced library and the source photos on the same volume why are you using a reference library? As you've been warned referenced libraries are not a strong point of Photos.

When you get this message when trying to use a photo

select Find Original and go and find/relink that photo. It may reset the entire library. If that is successful I strongly recommend you select all items in the library and use the File ➙ Consolodate… menu option to turn the library into a managed library.

Reply

Answer 2

May 6, 2021 10:25 PM in response to jweise

This is one of the problems with referenced files.

It is not the drive name that is used, but the volume ID (EG Disk0s2). This will change as you move the drive from one machine to another, and the link to the files will be lost.

As you have found, photos possesses no tools to efficiently reconnect disconnected referenced files. See this user tip from Leonie.

https://discussions.apple.com/docs/DOC-250002591

Reply

Answer 3

May 6, 2021 10:48 PM in response to TonyCollinet

Update - having done a bit of looking around, I have to confess I am not sure what form of volume ID photos is using to link the referenced files. I just know the link is lost when the volume is moved.

Reply

Answer 4

jweise Author

Level 1

9 points

May 7, 2021 5:37 AM in response to TonyCollinet

Thanks for the reply, Tony. I have found that there are a couple different identifiers for the disk. In the Photos.db (v4) there is the diskUuid in the RKVolume table. It looks like AAD0C510-DF79-38C8-8816-E72AE7E9A80F (in my case) and it matches what is called File system UUID in the Disk Utility application (info). I believe this number travels with the volume on the physical drive, as long as it isn't reformatted. And as you mention, there is the identifier (e.g. Disk0s2) which is the BSD device node, or disk + partition number, according to Disk Utilities. This number definitely has the potential to change when moved from one computer to another, or I believe, even if mounted in a different order on the same computer. I have yet to find a place where Photos stores this identifier. I was thinking of doing a test to see if keeping it consistent makes a difference.

Reply

Answer 5

jweise Author

Level 1

9 points

May 7, 2021 5:50 AM in response to jweise

I've noticed that Photos sets a "missing" flag in its internal sqlite database once you try to open a file that it thinks is missing. If the diskUuid/volume identifier and the path are accurate, and permissions are fine, what logic is Photos using that causes it to turn on the "missing" flag internally? What does it think is wrong, when everything appears on the surface to be right?

Brainstorming...

Some sort of internal userid that is different on two computers even if the username is the same?

Some sort of internal machine identifier that must be consistent?

Whatever it is, it would have to be stored as data somewhere available to Photos in order for it to know that the new situation is different than the old situation. The only information carried over from the first computer to the second, is stored within the Photos library, or in the photo files themselves (or xattr on file or folder or maybe volume).

I wonder if the logic in Photos is just over protective rather than flat out broken. Maybe it is checking several parameters and if anything is out of place it throws the flag. This could be to prevent users from unintentionally or I suppose even maliciously creating connections. Like, if you have a carbon copy clone of the drive with the photo files on it as a backup, you wouldn't want Photos to accidentally use it. That would be a different chaos.

Reply

Answer 6

léonie

Level 10

195,213 points

May 7, 2021 11:13 AM in response to jweise

Aperture has been a professional application and had elaborate tools to manage a referenced library. Photos for Mac allows us to reconnect referenced files, but Aperture's tools to do it quickly, are missing.

Probably Photos is supporting the referenced files only, so we can migrate referenced libraries to Photos, but should then instantly consolidate the library.

I tested the reconnecting a few times. It has been working well in the previous versions of Photos up to Photos 4 on Mojave. The big problems started with Catalina. Previously it sufficed to select the folder with the referenced files in the reconnect dialog, and Photos would reconnect all referenced files in the same folder at once, as long as we did not rename the folders or the files. I am surprised it is not working on your system. When you reconnect a missing original, are you selecting the file to reconnect or the enclosing folder? Try to select the enclosing folder to reconnect all photos in this folder at once.

Reply

Answer 7

jweise Author

Level 1

9 points

May 7, 2021 6:00 PM in response to Old Toad

Hi Old Toad,

You are right, I'm well aware of the pitfalls of managing referenced files. I've still chosen it because I prefer not to let Photos shuffle my files around and hide them inside the Photos package. Originally, I had my library on the system drive, and my photos on the external drive, but my Photo Library has grown to 458GB, and no longer fits on my system drive. So I moved the library to the external drive as well. There wasn't a pressing reason to consolidate, because everything was working. In the past, I have had success with the method you recommend of reconnecting all files in one fell swoop, but that isn't working this time. This time, it seems to only work per folder of files, and I have thousands of folders. Since I don't really want my files inside of the Photos package, I'm more likely to move on to a different solution than consolidate. However, I thought if I could find a way to reconnect the referenced files, that would allow me to complete my migration to a new computer and buy me some time to explore my options.

Reply

Answer 8

jweise Author

Level 1

9 points

May 7, 2021 6:11 PM in response to léonie

Hello léonie,

Thanks for the reply. I miss Aperture. I was happy with it. I think you make a good point that referenced files might only be supported in order to enable that migration. I also agree that reconnecting all at once worked when I first moved to Mojave. That's why things have been working for me in Photos 4 on Mojave on the machine I've been using for the last few years. Except for the fact that the problem I describe shows up in my tests between two different Mojave machines, as well as between Mojave and Big Sur. I'm in this strange window of time here where I have 3 computers, which is really helpful for doing tests, but the goal it get down to 1. I have tried the trick of reconnecting the enclosing folder, in fact, I've tried all levels in the folder hierarchy without success so far. It makes me wonder if I might have better luck if I did move all of my files to a new drive. Maybe it would trigger a different behavior than what I am seeing when I move the physical drive. I appreciate the thoughts.

Reply

Answer 9

jweise Author

Level 1

9 points

May 7, 2021 6:41 PM in response to jweise

I copied the library and the referenced files to an entirely different drive (and unmounted the original drive to avoid potential for file path confusion), and then when I chose the enclosing folder, all of the file references were fixed at once. This only worked with Photos 4.0 on Mojave. It doesn't work with Photos 6.0 on Big Sur (not very surprisingly).

Reply

Apple Photos says referenced files are "missing" after moving external drive to another machine

Similar questions