jonttaylor

Q: Time machine missing files

Timeline

 

1. 11 days ago I buy a new 15" retina haswell macbook pro running mavericks, and a new 2TB time capsule.

2. Yesterday I returned the macbook pro because it was making creaking noises and had it replaced with a new one.

3. Now I try and restore my backup.

 

Story

 

I restored from my latest time machine backup by pressing CMD-R while booting and specifying to restore from the Time Capsule. It took a couple of hours over wifi and finally restored, when I came back to it, the machine had rebooted and I was at my normal login screen.

 

So I thought the restore had been successful at this point and I log in, however it asks me for my Apple ID login as if i've never entered it before (I have), and I have some new mavericks icons at the end of my dock that I had previously removed. I open up my terminal (i'm a web developer) and I notice its missing my custom prompt (oh-my-zsh) which I track down to the .zshrc file missing.

 

I check my databases and they are all corrupted. The MySQL error file is constantly spamming itself saying the data is corrupt and that it might be missing its binlogs. A number of my apps don't remember that they are licensed. The machine has forgotten most of my preferences such as tap to click, however it has remembered things like my desktop background and my safari preferences (they were not synced from icloud I made sure).

 

I dig into this a bit more by mounting the time capsule and having a look around in the terminal.

 

cd /Volumes/Time\ Machine\ Backups/Backups.backupdb

ls -lah 2013*/Macintosh\ HD/Users/jon

 

This prints out the contents of what the backup considers my home folder to be like at every backup (there were 23 backups stored over the 10 days). First backup being on the 2nd of November, most recent being on the 12th November.

 

I see that from the 7th my .zshrc went missing, it was not in any subsequent backups after that, so I tried restoring back to that date. After restoring my terminal preferences are back, but the databases are still corrupt and I notice other things missing instead.

 

I dig into it a bit more and list the files again.

 

Then I notice just how bad it is.

 

On certain backups it has missed massive numbers of files, including directories such as Downloads Library Desktop etc. Then they come back in the next backup, or go missing forever. I know time machine doesn't backup the entire computer each time, but I know that it should be creating HARD links in the filesystem, so that EVERY backup appears to be a complete system backup. This is not the case.

 

I can rebuild this new macbook pro to have everything I need on it again, but I CANNOT trust my £250 Time Capsule to take backups, in which case it becomes an extremely expensive router + hard drive.

 

I think this might be a bug in Mavericks, 10 days is not very long for backups to start going astray!

 

Is there any way I can repair this somehow? Has anyone else had this kind of problem? I phoned apple support and they just tried to guide me through doing a re-install, when I explained that the backups themselves seemed to be missing files the support agent just didn't seem to understand, not very helpful at all. As a fellow engineer I would prefer to speak to another engineer and I could provide all the diagnostics they could want, short of giving them ssh access.

MacBook Pro with Retina display, OS X Mavericks (10.9), Backing up to a Time Capsule

Posted on Nov 13, 2013 1:52 AM

Close

Q: Time machine missing files

  • All replies
  • Helpful answers

first Previous Page 4 of 4
  • by lieut_data,

    lieut_data lieut_data Sep 30, 2015 5:25 AM in response to rationull
    Level 1 (0 points)
    Sep 30, 2015 5:25 AM in response to rationull

    I can confirm that my second TM backup was /not/ missing the files. (I don't know if it's missing any other files!)

  • by steveohm,

    steveohm steveohm Oct 11, 2015 2:25 AM in response to jonttaylor
    Level 1 (4 points)
    Desktops
    Oct 11, 2015 2:25 AM in response to jonttaylor

    Having same problems. With mine, I'm restoring to a new drive from a TM backup (Mavericks) and finding several non apple applications including all Microsoft applications were not restored. Nor was iphoto, but more importantly who knows what else. Currently trying previous backup instances, but I'm definitely frightened. This is my girlfriends personal machine, but to think this backup system is what I'm using on my own business machine, on which EVERYTHING is relying.... Totally terrifying. Did I miss something, or have Apple reps not chimed in at all on this serious issue? A lot of us don't have the skills or the patience to be running terminal commands on are gulag basis to confirm that TM is doing what it says it will. I'm ordering a new backup drive tomorrow and going back to a nightly disk clone. The time machine concept never quite added up to me. I far prefer a consistent single reliable complete backup than N number of incomplete backups over the last N number of weeks.

  • by Loner T,

    Loner T Loner T Oct 11, 2015 5:05 AM in response to steveohm
    Level 7 (24,002 points)
    Safari
    Oct 11, 2015 5:05 AM in response to steveohm

    TM has gotten better, but it still has software bugs. . It is better to have two different backup tools and offsite storage as the criticality and value of data increases.

  • by steveohm,

    steveohm steveohm Oct 12, 2015 11:30 AM in response to steveohm
    Level 1 (4 points)
    Desktops
    Oct 12, 2015 11:30 AM in response to steveohm

    UPDATE! I wish I could edit my first post. In my particular situation it turned out to be user error. I was able to copy over all the basic mac applications and utilities from another system - enough to be able to launch TM and see that my partner actually did have Applications excluded from her TM backups, though she swears she didn't set that. Oops. :/

  • by Loner T,

    Loner T Loner T Oct 12, 2015 12:05 PM in response to steveohm
    Level 7 (24,002 points)
    Safari
    Oct 12, 2015 12:05 PM in response to steveohm

    As long as you get all your files back, it is all good.

  • by Anne USB 1.1,

    Anne USB 1.1 Anne USB 1.1 Oct 28, 2015 1:52 PM in response to jonttaylor
    Level 1 (0 points)
    Oct 28, 2015 1:52 PM in response to jonttaylor

    I have the same problem. Soooo frustrating!

     

    I have been doing regular backups onto an external hard drive, which I only connect when I need to, so it's not connected permanently. I did a clean install of El Capitan the other day, then manually restored folders from the TM backup (I actually copied and pasted them via the Finder window). I only wanted to restore Documents, Pictures, Music as I decided to do a clean install of all my Apps as well (it wasn't that much of a big deal to do this, I don't have that many apps outside of the usual - Office, Firefox, etc.).

     

    All looked great until my boyfriend noticed missing documents. I didn't really believe him at first, I thought he must have deleted them before the last backup. Of course he hadn't - Time Machine messed up!

     

    I did a bit more investigating and seem to have identified a bit of a pattern:

    • New files that have been added to existing folders are being backed up
    • New folders and their contents are missing from the backup
    • Folders that have been renamed are missing their content (the folder itself is there but it's empty)

     

    I am not entirely certain about this though. It feels like Time Machine is getting confused between what has and hasn't changed?!

     

    This line in Pondini's post http://pondini.org/TM/Works.html makes me think Time Machine is sort of only half doing its job. Is it treating renamed folders as "nothing has changed"?

    If nothing in a folder is changed, Time Machine makes a single hard link for the folder, not one for each file inside it.


    Ugh, I am really upset about this. My boyfriend is quite angry with me because "my careless action" (i.e. not doing another kind of backup outside of Time Machine) has lost him a bunch of files that he can't retrieve from anywhere. *sigh*


    Definitely lost my faith in Apple, don't think I'll stick with them for my next laptop purchase. Shame.

  • by Brian Kendall,

    Brian Kendall Brian Kendall Nov 17, 2015 2:57 PM in response to Brian Kendall
    Level 1 (5 points)
    Nov 17, 2015 2:57 PM in response to Brian Kendall

    I'd like to make an update about my situation...

     

    I decided to give Time Machine a try again, and in doing so wrote a python script to specifically check to see if it was missing any important files of mine. The tool scans the entire filesystem and then compares against Time Machine's backup, and then reports back any files that are missed. If the files are marked as being excluded (as reported by using 'tmutil isexcluded path-to-file') it will list them separately. It then gives the option to remove the exclusion marker from the excluded files.

     

    It turns out all of the important files of mine that Time Machine was missing were marked as being excluded. After I ran this tool, unmarked those files, and did a Time Machine backup, everything was once again backed up properly. I'm hoping this will fix the issue for me and allow me to use Time Machine once again, because when it's working it simply is the best tool available for making local backups.


    (I suppose it's also possible that starting again from scratch or all of the OS updates that have happened between then and now could have fixed the underlying issue, but there's no way to be sure at this point.)

     

    Would any of you in this thread consider trying out my script? I'm very curious if the files that people are concerned about being missing are in fact just being marked as excluded, and Time Machine is actually functioning as it should (for some definitions of functioning). Here's a link to my github page: https://github.com/briankendall/check-time-machine

     

    You simply run the script as root from Terminal, i.e.:

    sudo python checktimemachine.py

    - or -

    sudo python checktimemachine.py /path/to/specific/directory

     

    and then wait for a bit while it scans everything. It's programmed to automatically ignore intentionally excluded operating system files (from Apple's private list), as well as several directories and files that are marked as excluded that I was pretty sure were safe to ignore. These exclusions are listed at the top of the script and you can modify them as you want.

     

    Note that it requires the Time Machine volume to be present and accessible from the system... not sure how this will work with Time Capsule or networked Time Machine volumes, but in theory it should work.

  • by billyearnest,

    billyearnest billyearnest May 31, 2016 10:41 AM in response to Brian Kendall
    Level 1 (4 points)
    May 31, 2016 10:41 AM in response to Brian Kendall

    I recently realized that my time machine backups are missing files. REAL FILES. WORKING & CURRENT PROJECT FILES. UNACCEPTABLE!!!!!!

    Yosemite with all updates.

    I'm going to try your routine if I can figure out how to.

    Thanks,

    Billy Earnest

first Previous Page 4 of 4