14 Replies Latest reply: Apr 28, 2010 3:06 PM by Espen Vestre
Espen Vestre Level 1 Level 1 (140 points)
Hello,

today I exchanged my MacBook Pro's built-in 320GB hard disk with a 500GB one. As a preparation, I had bought a brand new 1.5TB disk for backup yesterday, and started a fresh TM backup on that disk. Today I then installed Snow Leopard on the new 500GB disk and restored from the TM backup. I got suspicious as I noted that the amount of data after the restore was too low, and started figuring out what would be missing. I was very surprised and disappointed to discover that my iPhoto library wasn't on the TM disk at all - on NONE of the backups!

The log on the TM disk says "Node requires deep traversal: /Users/<me>/Pictures/iPhoto Library reason:must scan subdirs|missed reservation|"

- but doesn't report any errors.

Very, very disappointing. I think I'll redo the restore from the old disk, I don't know what else may be missing, so...

MBP (late 2008) 15" 2.8/4/320 - MBP 15"/MBP 13"/MB 13"/PB 12" - iPhone 3G, Mac OS X (10.6.2), At work I use a Hush with Ubuntu 64 bit
  • 2. Re: Time Machine just forgot about my iPhoto Library!!
    Espen Vestre Level 1 Level 1 (140 points)
    Thanks, but I knew about all that.

    I'm now waiting for the slow (2.5" disk via USB) from the original disk, but I'm beginning to think that it wasn't really necessary, because I found a file which may be the start of an explanation of what happened:

    sh-3.2# ls -l */.reservations.plist
    -rw------- 1 root _unknown 286 Dec 2 11:58 2009-12-02-115839/.reservations.plist
    -rw------- 1 root _unknown 286 Dec 2 12:41 2009-12-02-124101/.reservations.plist
    -rw------- 1 root _unknown 286 Dec 2 13:41 2009-12-02-134144/.reservations.plist
    -rw------- 1 root _unknown 286 Dec 2 14:41 2009-12-02-144151/.reservations.plist
    -rw------- 1 root _unknown 286 Dec 2 15:41 2009-12-02-154121/.reservations.plist
    -rw------- 1 root _unknown 286 Dec 2 16:40 2009-12-02-164056/.reservations.plist
    -rw------- 1 root _unknown 286 Dec 3 10:00 2009-12-03-100057/.reservations.plist
    -rw------- 1 root _unknown 286 Dec 3 10:37 2009-12-03-103716/.reservations.plist
    -rw------- 1 root _unknown 286 Dec 3 10:37 Latest/.reservations.plist

    This file (identical in all backups, note that 2009-12-02-115839 was the first, full backup) looks like:

    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
    <plist version="1.0">
    <dict>
    <key>reservedPaths</key>
    <array>
    <string>/Users/**my username censored**/Pictures/iPhoto Library</string>
    </array>
    </dict>
    </plist>

    Does anybody know what this file is for? It's not the same as the exclusion list, and I never asked for an exclusion anyway.
  • 3. Re: Time Machine just forgot about my iPhoto Library!!
    Espen Vestre Level 1 Level 1 (140 points)
    I searched with Google and I searched all of ADC, and found NO TRACES of .reservations.plist or the reservedPaths plist key. The backup of my most important files killed by the most undocumented feature in the world?? I still have my previous TM disk, with a fresh backup from yesterday, and I exchanged my HD while it was still working, so I won't miss a file, but this still really makes me angry.

    And here's what I just found out: My old backup disk has a lot of .reservation.plists too, but not in EVERY backup directory. In fact, it probably only has it in the backups for those occasions where iPhoto wasn't running... and now look what I found:
    http://www.pigsgourdsandwikis.com/2009/04/time-machine-vs-iphoto.html

    This is really a disappointment. If TM wants me to quit iPhoto before it can backup, why doesn't it tell me so during the initial backup? Or even during the subsequent ones, if it discovers that some days has passed since the previous backup of iPhoto Library?
  • 4. Re: Time Machine just forgot about my iPhoto Library!!
    Espen Vestre Level 1 Level 1 (140 points)
    And here's what I just found out: My old backup disk has a lot of .reservation.plists too, but not in EVERY backup directory. In fact, it probably only has it in the backups for those occasions where iPhoto wasn't running... and now look what I found:


    Oops, too many negations, of course I meant: "...it probably only has it in the backups for those occasions where iPhoto WAS running".
  • 5. Re: Time Machine just forgot about my iPhoto Library!!
    macjack Level 9 Level 9 (50,510 points)
    Never mind that didn't work.



    -mj

    Message was edited by: macjack
  • 6. Re: Time Machine just forgot about my iPhoto Library!!
    Espen Vestre Level 1 Level 1 (140 points)
    I've filed a bug report with Apple and have made a habit of quitting iPhoto every now and then to make sure I get at least one daily backup.
  • 7. Re: Time Machine just forgot about my iPhoto Library!!
    direwolf8 Level 4 Level 4 (1,280 points)
    I don't know of any backup program that will reliably backup open files. If you want proper backups, leave the computer on and exit all applications.
  • 8. Re: Time Machine just forgot about my iPhoto Library!!
    Pondini Level 8 Level 8 (38,720 points)
    Yes, it does appear that Time Machine will not back up the iPhoto "package" while it's open. That's what all those "reservation" plists are about. You'll also see a message in your system log when it does get backed-up, about a "deep traversal" being required because of a "missed reservation."

    And yes, it certainly should send a message, at the very least!
  • 9. Re: Time Machine just forgot about my iPhoto Library!!
    Espen Vestre Level 1 Level 1 (140 points)
    direwolf8 wrote:
    I don't know of any backup program that will reliably backup open files. If you want proper backups, leave the computer on and exit all applications.


    This is very bad advice, and absolutely contrary to the whole idea of Time Machine. Exit all applications every hour, you say? Fortunately, Time Machine is able to take proper backup of most files, open or not, it depends on the state of the files. Of course e.g. virtual machine files or many kinds of databases (including iPhoto) can be in an inconsistent state, but there's no reason for iPhoto to be in an inconsistent state when it's just idling.
  • 10. Re: Time Machine just forgot about my iPhoto Library!!
    nolamike Level 1 Level 1 (5 points)
    Just posting to relate similar experiences. This must get resolved somehow. A message saying "the following items not backed up" is contrary to Apple's (and Time Machine's) "don't bother the user unless absolutely necessary" philosophy, but in this case, lacking a better solution, OK, bother me.

    In my case, I was surprised at how little space my fresh backup took, then realized that indeed my 205GB iPhoto library was missing. I quit iPhoto and waited several hours, and it would not put it into the backup. After a little forum reading, I was ready to wipe my backup drive, ensure iPhoto was closed, and try again, but then I remembered an old trick for another TM issue: Exclude the iPhoto LIbrary (under "Options..." in Time Machine), run a backup, then include it again. Sure enough, it is now being added to the backup as I type.

    But, to reiterate, this is really distressing. If I had to identify the group of files in my life I would most hate to lose, my photos would probably be it. I use two TM backup disks that rotate between being in use and sitting in my desk drawer at work, to ensure my files are safe. [By the way, if you don't have an offsite backup, consider the possibility of fire or theft, which in a flash could wipe out your original and your backup.]

    In general, Time Machine is an awesome application and concept, but I'm glad I'm not on the receiving end of certain support calls Apple must be receiving these days.
  • 11. Re: Time Machine just forgot about my iPhoto Library!!
    Almarmar Level 1 Level 1 (5 points)
    Another one with the same problem, but with a small difference: when I have iPhoto opened and a Time Machine backup starts, what Time Machine decides is to backup the entire iPhoto library!! Mine is 68GB full of photos and some videos!!!! Try to realize how long it takes to backup that amount of data, and how funny is to discover that, two months ago, my oldest backup was one year old, and now, as the 1TB Time Machine is full of iPhoto garbage, the oldest backup is from november 2009!! Just two months ago!! I've lost all previous backups forever!! Great job Apple!! Another reason to migrate all my photos to Picasa. I'm having more and more reasons every day, as iPhoto becomes more and more slow and do things like this with backups...

    I'm really dissapointed with iPhoto. My biggest illusion before buying my first Mac, and my biggest deception after it. Some reasons:

    -Geolocation only works for itself, if you want to export your photos to Picasa, you have to geolocate them again.

    -Events only works for itself: even in the iPhone, they don't exist. You have folders instead. Oh, yes, and faces, that thing very funny to play with the first days and that almost everybody forgot about long time ago.

    -Extremely slow RAW processing.

    Etc...

    EDIT: I just edit to correct one information. I have just verified the logs and the last deleted backup is fabruary the 9th, 2010!! Two days ago!! Even funnier!!

    Message was edited by: Almarmar

    Message was edited by: Almarmar
  • 12. Re: Time Machine just forgot about my iPhoto Library!!
    Almarmar Level 1 Level 1 (5 points)
    Hello,

    Some days ago I found something that I tried, and seems to work. At least, I don't have more "iPhoto full backups" after this thing. I'll try to explain it:

    I booted from the install disc, and from the utility menu, chose reset password for Macintosh HD, and at the bottom of the box clicked Reset Home Directory Permissions and ACLs. Then, rebooted again from the HDD, and done.

    Why I did this? Because I saw exactly the same ACL error messages as in this post: http://www.mac-forums.com/forums/os-x-operating-system/167580-successfully-remov ed-acl-found-but-not-expected-warnings-permissions-repair.html
    And in some other place, I read that our iPhoto problem is related to the permissions and some systems files that tell to Time Machine which photo is backed up and which doesn't. And as this solution reset all home directory permissions, I decided to try. The only consequence of this solution is that all spotlight data of your computer is deleted and indexed again, but I think it worth the try , as you don't loose anything.

    I'm not sure if it has 100% solved my trouble, but since I did it one month ago, Time Machine seems to be working again as expected. I wish it
  • 13. Re: Time Machine just forgot about my iPhoto Library!!
    Espen Vestre Level 1 Level 1 (140 points)
    Almarmar wrote:
    Hello,

    Some days ago I found something that I tried, and seems to work. At least, I don't have more "iPhoto full backups" after this thing. I'll try to explain it:

    I booted from the install disc, and from the utility menu, chose reset password for Macintosh HD, and at the bottom of the box clicked Reset Home Directory Permissions and ACLs. Then, rebooted again from the HDD, and done.

    Why I did this? Because I saw exactly the same ACL error messages as in this post: http://www.mac-forums.com/forums/os-x-operating-system/167580-successfully-remov ed-acl-found-but-not-expected-warnings-permissions-repair.html
    And in some other place, I read that our iPhoto problem is related to the permissions and some systems files that tell to Time Machine which photo is backed up and which doesn't. And as this solution reset all home directory permissions, I decided to try. The only consequence of this solution is that all spotlight data of your computer is deleted and indexed again, but I think it worth the try , as you don't loose anything.

    I'm not sure if it has 100% solved my trouble, but since I did it one month ago, Time Machine seems to be working again as expected. I wish it


    Thanks for your advice, very interesting!

    I have some new information in this case:
    I just moved all my data to a new MacBook Pro from a TM backup of the old one, only to discover that I AGAIN lacked 5 weeks worth of iPhoto images! And this time it's more complicated, since I know that iPhoto hasn't been running all the time. In fact, before I did the restore to the new machine, I restarted the old machine and ran a backup.
  • 14. Re: Time Machine just forgot about my iPhoto Library!!
    Espen Vestre Level 1 Level 1 (140 points)
    Some more info:

    In addition to stop backing up iPhoto more than a month ago, TM had also failed to back up the following:

    - one single file which was replaced more than a month ago
    - additions to my trial Aperture 3 library done 10 days ago
    - at least one music file from my iTunes library (which, at some point, was moved by iTunes from one folder to another)

    It's a bit late to restore completely from the old mac in target mode, so I've now done a lot of command line stuff (du, find, diff etc.) in order to spot differences, and I think the above must be all. But this has been a very disappointing experience.