1 2 Previous Next 29 Replies Latest reply: Jan 2, 2014 10:34 AM by Old Toad
Pascal Mah Level 1 Level 1 (0 points)

I have a huge iPhoto library containing 40 000+ images, as well as a large amount of metadata (comments, Faces,...). Since I need to read-access the image files with other software, I use a referenced library.

 

Everything was fine until the day I decided to upgrade for a larger HD. So I simply restored full content of my old drive on the new drive using a TimeMachine backup. Nothing else was modified (no directory structure changes, not even the volume name).

 

Since then, the problems. Essentially, references to the original files are lost: Thumbnails are OK, but upon accessing the larger view, iPhoto asks me to locate the file... After having done so, other pics would load nice, but nothing would stick upon a quit and relaunch of iPhoto... (Funnily, this happens only for pics that were imported *after* I upgraded from iPhoto '08 to iPhoto '11).

 

I tried many things: rebuilding the database, rebuilding the database on the older drive prior to backup, etc... Nothing would help.

 

So, my questions:

 

* Whad did I do wrong?

 

* Is there a way to "repair" file references? (That should be a fairly obvious functionality of database rebuilding)

 

* Is hacking into the Library.apdb file part of the officially supported solutions?

 

Obviously, recreating a new iPhoto library is not an option for me, due to the amount of metadata in my library that I don't know how to transfer. Also, I don't consider upgrading to Aperture, which is just too complex and loaded of features which are totally useless to me.

 

Running Lion and latest iPhoto update.

 

Thanks!


iPhoto '11, Mac OS X (10.7)
  • 1. Re: iPhoto '11 referenced library problems
    Terence Devlin Level 10 Level 10 (123,855 points)
    * Whad did I do wrong?

     

    Not sure, but something in the path to the files changed.

     

    * Is there a way to "repair" file references? (That should be a fairly obvious functionality of database rebuilding)

     

    With iPhoto, afraid not.

     

    * Is hacking into the Library.apdb file part of the officially supported solutions?

     

    No. But if you manage, plese post details on how you did it. This question comes up a lot.

     

    You know that all of the metadata - with the exception of Faces - can be written to the files.

     

    Regards

     

     

    TD

  • 2. Re: iPhoto '11 referenced library problems
    Pascal Mah Level 1 Level 1 (0 points)

    YES!!!

     

    I finally managed to recover my iPhoto library!

     

    But it was not easy. I had to hack into the database file to put up things right.

     

     

    After a significant amount of trial-and-error, here is the procedure I came to, which finally worked for me (use at your own risk):

     

    0. Make sure you make enough backups so as to revert to the previous state if something goes wrong!

     

    1. Make a copy of your iPhoto Library [Show Package Contents]/Database/apdb/Library.apdb to your desktop.

     

    This file contains most of the data involved for managing your iPhoto data.

     

    2. Open this file using a SQLite database manager.

     

    I used Navicat 9.1, which has nice import-export features. For direct editing, Base 2.0 is may be easier.

     

    3. Open the table RKMaster.

     

    This table contains all records for the individual pics of your library.

     

    4. Correct the path of each of your pic files in the imagePath column to their current filepath.

     

    This contains the path of your pic files at the time they were imported, and is not updated by iPhoto even though you have moved your pic files somewhere else.

     

    If you have several thousands of paths to correct, a good idea is to export this column to a text file (also including the modelId column for referencing), and to correct paths using the substring Search&Replace function of your favorite text editor. Be sure to re-import the corrected data properly using the modelId column as references.

     

    5. If needed, correct in the same manner the content of the fileVolumeUuid column to the value of the drive currently containing your picfiles.

     

    If needed, obtain this value considering a pic which was recently imported from that drive.

     

    6. If your drive name was modified, also correct its name in the name column of the RKVolume table.

     

    Identify the proper record by considering its uuid previously obtained.

     

    7. If you're happy with your work, quit the database managing program and put Library.apdb back in its original location inside your iPhoto Library.

     

    Keep the older somewhere in case something goes wrong.

     

    8. Run iPhoto to see if your work is successful!

     

    At this point, you might consider rebuilding the Photo Library (hold alt-cmd while launching iPhoto) and choose Repair the iPhoto Library Database (Be sure to leave the Rebuild the iPhoto Library Database from automatic backup unchecked!). This might correct some possible unconsistencies resulting from your edits. Also, a good thing might be to rebuild all thumbnails. If iPhoto does not bug you for locating files during that process, you have possibly done your work right! If not, back to step 1.

     

     

    As said, this worked for me, with no visible inconsistencies or side effects to be noticed in the behavior of iPhoto (at least for now). But possibly some knowledgable people might comment and improve that process, and some database script guru might also help automating this. Please comment.

     

    Lessons learned (How I understand the things as far as I know):

     

    A. Library.apdb stores the initial drive and path of the pic files at the time they were imported. This is not modified as the files are moved around.

     

    B. Some other data (BLOBs?) is used to track the files at their actual location. Therefore, it seems OK to move the pic files after they were imported.

     

    C. Unfortunately, this other data is broken if the file is recreated (even with same content and location), for example with a file-based backup and restore (Time Machine).

     

    D. In that case, iPhoto is unable to recover the file if its current drive and path does not match the one when the file was imported, stored in the database. Also there is no mechanism in iPhoto to correct this data.

     

    E. Therefore, it is very important to import pic files in iPhoto only when these are already in their final location! If not, your iPhoto Library won't survive a TimeMachine backup and restore! (Possibly, a block-based drive backup could work... I don't know).

     

    F. ...And Apple should really, really fix this issue! (By correcting the stored filepath and drive to the current file location at least when the database is repaired, and by offering at least some basic file reconnection options).

     

     

    Hope this helps...

     

    - Pascal

  • 3. Re: iPhoto '11 referenced library problems
    PartisanEntity Level 1 Level 1 (15 points)

    Thanks very much for posting this, it saved me from having to reimport and sort 12000 photos! Worked like a charm.

  • 4. Re: iPhoto '11 referenced library problems
    PartisanEntity Level 1 Level 1 (15 points)

    Update: it worked but only to a point. For some reason I had my shortcut files in my iPhoto library. In such cases this method will not work for obvious reasons.

     

    I am not sure why I had these shortcut files, it could be due to previous versions of iPhoto. So I had to bite the bullet and create a new iPhoto library and re-import all my photos.

  • 5. Re: iPhoto '11 referenced library problems
    robig1 Level 1 Level 1 (0 points)

    Worked fine

     

    I was using the sqlite console client using a query like this to update my database entries to move all files on the external Volume to the local drive:

    update RKMaster set imagePath = replace(imagePath,'Volumes/ExternalVolume','Volumes/<Destination>')   where imagePath like '%ExternalVolume/%';

     

    Thank you.

     

    robert

  • 6. Re: iPhoto '11 referenced library problems
    michaelab Level 1 Level 1 (15 points)

    Pascal Mah wrote:

     

    Lessons learned (How I understand the things as far as I know):

     

    A. Library.apdb stores the initial drive and path of the pic files at the time they were imported. This is not modified as the files are moved around.

     

    B. Some other data (BLOBs?) is used to track the files at their actual location. Therefore, it seems OK to move the pic files after they were imported.

     

    C. Unfortunately, this other data is broken if the file is recreated (even with same content and location), for example with a file-based backup and restore (Time Machine).

     

    D. In that case, iPhoto is unable to recover the file if its current drive and path does not match the one when the file was imported, stored in the database. Also there is no mechanism in iPhoto to correct this data.

     

    Regarding point B, yes, for referenced files (ie, image files that are stored outside the iPhoto Library folder structure, when you have the "Copy Images to Library" option switched off), iPhoto '11 stores the file location in a BLOB column in RKMaster called fileAliasData. The alias blob data is basically the contents of a plist file which has in it the serialized data of a FileAlias object which is a proprietary object type that iPhoto uses so we don't know it's structure. However, if you view the data you can see, amongst the binary garbage characters, the file path that is being stored. It should be possible to reverse engineer the FileAlias format to be able to write a program to 'fix' or update this data, a project I might think about

     

    Prior to iPhoto '11, iPhoto used regular OS X file aliases within the iPhoto Library folder structure (under the Originals folder, which in iPhoto '11 is now called Masters) to store the location of referenced files. This is the reason that Alias Herder no longer works for iPhoto '11, because it no longer uses normal file aliases.

     

    Michael.

  • 7. Re: iPhoto '11 referenced library problems
    michaelab Level 1 Level 1 (15 points)

    PartisanEntity wrote:

     

    Update: it worked but only to a point. For some reason I had my shortcut files in my iPhoto library. In such cases this method will not work for obvious reasons.

     

    I am not sure why I had these shortcut files, it could be due to previous versions of iPhoto. So I had to bite the bullet and create a new iPhoto library and re-import all my photos.

    Yes, the shortcut (or alias) files in the iPhoto Library are from an earlier (pre '11) version of iPhoto. That is the way iPhoto used to store file references.

     

    I was able to copy all the real images over my shortcuts and then update the RKMaster table in the iPhoto database accordingly so that it all worked. This way I managed to consolidate my library into the iPhoto Library folder and got rid of any references which could cause be trouble in the future.

     

    Michael.

  • 8. Re: iPhoto '11 referenced library problems
    ddewan Level 1 Level 1 (0 points)

    I have a similar issue, although I could not find old server being referenced anywhere in RKMaster. Any other suggestions?

  • 9. Re: iPhoto '11 referenced library problems
    LarryHN Level 9 Level 9 (56,675 points)

    With iPhoto 9.3 you can open your library with Aperature 3.3 which can resolve the reference issus - upgrading iPhoto and aperature or buying Aperature is a supported way of resolving this now

     

    LN

  • 10. Re: iPhoto '11 referenced library problems
    odysseus Level 3 Level 3 (910 points)

    Larry, what do you think will happen if I run "Consolidate Originals" in Aperture on my library which already has the originals in place because I used AliasHerder?

  • 11. Re: iPhoto '11 referenced library problems
    minethatbird Level 1 Level 1 (0 points)

    This solution worked great for me too with one caveat. In addition to setting the fileVolumeUuid column to match that for recent photos (which for me was null) I also had to change the fileIsReference column from 1 to 0. I was using iPhoto 9.4.2.

  • 12. Re: iPhoto '11 referenced library problems
    josefssonfredrik Level 1 Level 1 (0 points)

    I had a similar problem where the file path was wrong after changing the user name from josefssonfredrik to fredrikjosefsson, iphoto looked in the following folder .../Users/josefssonfredrik/Pictures/.. when it was suppesed to look in .../Users/fredrikjosefsson/Pictures/.. I solved it using this method:

     

    1. Create the folder wich the images was suppose to be in ...Users/josefssonfredrik/Pictures/...

    2. Copy all the images to this folder (this will result in that IPLM can find the images)

    3. Use IPLM and rebuild the library.

     

     

    This works if you can create the folder which the images was suppose to be in (there may be problem if the hdd drive name is different). Also if you can live with the fact that all images are copied inside the new rebuild iphoto library.

  • 13. Re: iPhoto '11 referenced library problems
    tyler_khh Level 1 Level 1 (0 points)

    I had this exact same problem. I had backed up my referenced library to an external drive, reinstalled OS X on my MB Pro's internal, and then copied the entire library back to the internal drive with the exact same path as before. Even though it was the same drive and the same path, iPhoto could "Reveal in Finder" but not "Edit" (offline error).

     

    EASY SOLUTION for me..... During my reinstall, I had named the main disk "MAC". On my old installation I had named it "TYLER". I simply changed the name of the main disk back to the old name "TYLER" and voilà iPhoto worked normally again. Literally a 2-second fix!!!!!

     

    Hopefully this helps someone else out!

  • 14. Re: iPhoto '11 referenced library problems
    LarryHN Level 9 Level 9 (56,675 points)

    Even though it was the same drive and the same path, iPhoto could "Reveal in Finder" but not "Edit" (offline error).

     

    EASY SOLUTION for me..... During my reinstall, I had named the main disk "MAC". On my old installation I had named it "TYLER". I simply changed the name of the main disk back to the old name "TYLER" and voilà iPhoto worked normally again. Literally a 2-second fix!!!!!

     

    It was not the same path since you changed the hard drive name - when you correcte the hard drive name and thereby made the path the same again you were fine

     

    However there is no advantage at all to a referenced library and it creates many problems like (but not limited to) the difficulty in changing the path which makes upgrading hardware or replacing defective hardware difficult to impossible  --  Referenced libraries are strongly not recommended now

     

    LN

1 2 Previous Next