Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

UPDATED: Serious Time Machine bug on Mountain Lion

This is continued and summarized from a previous thread, Serious Time Machine bug on Mountain Lion, which has grown rather large and unwieldy (and which I marked Solved in error). There are a few other threads that touch on it as well.



Some of us have seen a few cases of a pretty nasty problem with Time Machine backups on Mountain Lion. It doesn't seem to affect a lot of users, but those who have it, don't know it. It seems to have started at different times for different users, some as long ago as December, 2012.


The backups cannot not be used to do a full system restore from recent backups -- they don't appear on the selection screen that shows only "complete" backups. Plus, they're not recognized as backups by Setup Assistant or Migration Assistant. They don't appear on the Select the Source or Select Your System windows at all. But your user data is fine via the Time Machine browser (the "Star Wars" display).


What's happening is, the top-level Applications, Library, System and (hidden) private folders are being excluded by some process (not the user, and the exclusions do not appear on the Time Machine Preferences window). But all backups complete normally, and Verify Backups from the Time Machine menubar on network backups, and Verify Disk or Repair Disk via Disk Utility, don't show a problem (because what's there is intact; those processes can't tell that what's missing isn't supposed to be missing.)


In addition, we've seen a few threads where the problem is intermittent -- backups are unexpectedly large, sometimes often, sometimes less so, intermittently. In those cases, the folders are backed up, then skipped, then backed-up again, then skipped, etc. Since they appear to be "new" when backed-up again, they're backed-up in full, making backups of 15 GB or more.



See #D10 in Time Machine - Troubleshooting for an easy way to tell if you're affected, and a fix.


However, as we have no clue as to the cause yet, we need to be sure Apple knows about this, in detail, while it's happening, so I'm asking for some folks who are having these problems, and are fairly comfortable with the Finder at least, and following directions to do other things you may not be so familiar with, to help us collect information and contact Apple before running the fix in the link above (as that will destroy some of the evidence).


We're especially interested in folks who are covered by AppleCare, as you can call them for free and get some fairly quick attention. We can supply some "ammunition" for when you call them (since I haven't had this problem, and don't know how to reproduce it, I can't report it myself).



If you're having this problem and just want to run the fix, by all means do so (and check it periodically), but please don't post back with just a "me, too" post, unless you have something significant to add -- this thread may get long and those won't really help.


But if you're having a different problem, please do not respond to this thread. Start a new one, instead. If you're not sure how to do that, please go to the main Apple Support Communities page; some Tutorials are listed at the right. They'll show you how to get started.

Posted on Jun 24, 2013 6:48 AM

Reply
171 replies

Jul 7, 2013 5:57 AM in response to Pondini

It appears I'm having the same problem. Thankfully, I didn't need my Time Machine backups to restore my machine (I was only after my data). I have upgraded my MBP to a dual HDD setup. I installed OSX on an SSD, backed up to both time machine on an Apple Time Capsule and also to a time machine set up on a ReadyNAS and then wiped the drive the backups were for (in order to remove the OS and the put the data back on). I thought it was odd that time machine was only backing up 280gb of a drive that was at least 450gb full, but when I looked through Time Machine everything appeared to be there.


Here's where the problems start. I wiped the drive and booted up into the SSD. I opened up Migration Assistant and it would only go as far as selecting the back up drive. It'd ask me to connect, give me a log in box and then sit and do nothing. I tried booting up from a bootable USB Mountain Lion disk and restoring data through time machine there, but it would not recognise and kept insisting it wasn't a bootable backup.

After some messing around in terminal I managed to run an associatedisk and access the data directly from finder and copy and paste things to my data drive (this is from the Time Capsule backup, not the ReadyNAS one). However, some folders didn't copy everything across (100gb or 146gb folder). Turns out the library folders didn't copy. I managed to get to them through terminal and run a cp -r to get them across. Finder previously had told me that these were 40gb folders. Upon copying them to my data drive, they turn into 400gb folders. I pulled out the stuff I needed and then deleted them (again, through terminal, because finder wouldn't let me).


So what I thought would be a simple and easy data migration process turned into nearly a full 24 hours of data copying and terminal hijinks to get my time machine to do what it was supposed to in the first place. Additionally, apart from not backing up most of OSX, it also didn't originally backup any of the applications (there's maybe two or three for each user been backed up, really random ones too).

Jul 7, 2013 7:11 AM in response to jmrz

jmrz wrote:


It's definitely not the TC's hardware. I have the same problem occuring on both the TC and the brand new ReadyNAS.

I was replying to cyrus_b_uk.


Note that since he's clearly having a different problem, I asked the Hosts to split his posts and my replies to a new thread. Unfortunately, your second reply got moved there, too.



How do you go about repairing the sparse bundle? I'd be interested to see if that fixes the problem.

See #A5 in Time Machine - Troubleshooting.


It's not clear from your other post -- were the top-level Applications, Libaray, and System folders backed-up or were they skipped entirely?

Jul 7, 2013 7:20 AM in response to Pondini

Pondini, ah apologies, didn't realise that!


Navigating through to the latest backup via terminal shows that the backup contains the User accounts (minus all but two/three applications in their respective applications folders) and a bunch of other folders (hidden in finder) e.g. cores, opt, etc, tmp, var.


I can't see any Applications outside of the user accounts. I can only see Library folders under the user accounts through terminal (these are the ones that increased in size dramatically when coping via terminal).

Jul 7, 2013 7:38 AM in response to jmrz

jmrz wrote:

. . .

Navigating through to the latest backup via terminal shows that the backup contains the User accounts (minus all but two/three applications in their respective applications folders)


If there were individual Applications folders inside user home folders, do you mean they weren't backed-up correctly?


and a bunch of other folders (hidden in finder) e.g. cores, opt, etc, tmp, var.


How about private? The bug addressed here skips it (that's why the backups can't be used with Setup Assistant or Migration Assistant).


I can only see Library folders under the user accounts through terminal (these are the ones that increased in size dramatically when coping via terminal).

That's probably the permissions issue -- the Finder won't show the size of things you don't have permission to read.


I can't see any Applications outside of the user accounts.

It does sound like you have/had this bug.


Unfortunately, you also don't have the evidence (it was in the top-level Library folder).


You might want to look back further in your backups -- you may have the Applications folder from before the problem started. If so, you can recover some "simple" apps by just copying from it. "Complex" apps, however (typically the ones that come with their own installers), also put files in various places in the top-level Library folder, and won't work properly, if at all, without them. Unless you can find and copy them, too, you'll have to reinstall those apps from the original discs or downloads. Most likely, any serial numbers or purchase keys will be gone, too, along with any Internet plug-ins, widgets, etc., installed for all users.



Once you get as much recovered as you can, and set up Time Machine again, run a backup and see if those top-level files get backed-up. If not, report it to Apple before doing anything else (ie, while you still have the evidence). If you're covered by AppleCare, that's the call to make. If not, and assuming you don't really want to pay them $50 or $60 for the call, you can file a detailed Bug Report. In either case, I'll be happy to show you how and suggest what to say.


Don't hesitate to ask for clarification, etc., and keep us posted.

Jul 7, 2013 5:16 PM in response to Pondini

The individual applications folders inside the user directories were not backed up correctly. I had three user accounts on that computer and each only had two or three applications in the backup (all non-apple ones).


In terminal, the folders showing in the latest backup folder are: Documents, Network, User Guides and Information, Users, Volumes, cores, etc, lost+found, mach_kernal, opt, tmp and var. There is definitely no private folder.


I don't understand how the Library folder for one user can go from 40gb to 400gb though. The drive it came from was only 500gb full and there's more than 100gb of data outside of that Library folder. I just grabbed my fonts and keychains and deleted the rest of it, space was then restored.


I'm not too worried about the applications, most of them I can drag and drop from my iMac or I have the install files/cd's on external storage.

I'm no longer covered by applecare (this machine is nearly a year out of it's applecare, hence the dual hdd upgrade), but I am more than happy to send in a bug report. Thankfully I was savvy enough to figure out how to get to my data, but I feel incredibly sorry for anyone who has had this problem and were not able to get around it.

Jul 7, 2013 6:01 PM in response to jmrz

jmrz wrote:


The individual applications folders inside the user directories were not backed up correctly. I had three user accounts on that computer and each only had two or three applications in the backup (all non-apple ones).

That's odd, but I've never tested much with Apps in user folders.


Were there Apple apps in those folders? Any pattern to what was skipped? I'd expect TM to back up anything it finds there.


In terminal, the folders showing in the latest backup folder are: Documents, Network, User Guides and Information, Users, Volumes, cores, etc, lost+found, mach_kernal, opt, tmp and var. There is definitely no private folder.

That's clearly this bug, then. 😟


I don't understand how the Library folder for one user can go from 40gb to 400gb though. The drive it came from was only 500gb full and there's more than 100gb of data outside of that Library folder. I just grabbed my fonts and keychains and deleted the rest of it, space was then restored.

Also odd, and doesn't seem related. Could be some sort of directory problem, I suppose. No way to tell now, of course.


I'm no longer covered by applecare (this machine is nearly a year out of it's applecare, hence the dual hdd upgrade), but I am more than happy to send in a bug report.

Cool. Keep an eye on your backups, to be sure those top-level folders do in fact get backed-up. For some folks, they're backed-up for a while, then skipped, then backed-up again, causing a large backup (15-20 GB or more). If it happens, post back and I'll give you info on how to report a bug and run a diagnostic package to attach to it, and also how to fix it (at least temporarily).


It doesn't seem to affect a large number of users, and seems connected to 3rd-party apps somehow, but we don't really know what's causing it. Something is changing the "plist" file where TM's instructions are kept, telling it to skip those folders. The way it's done is not something you can do via the TM Preferences window, and isn't shown there, so the only way to tell is to look at the backups.


Thankfully I was savvy enough to figure out how to get to my data, but I feel incredibly sorry for anyone who has had this problem and were not able to get around it.

Yup. 😟 I helped a couple of users do just that -- one very inexperienced, but she was also very determined -- and isn't inexperienced anymore!


Keep us posted.

Jul 11, 2013 7:41 PM in response to Pondini

Pondini wrote:


That's odd, but I've never tested much with Apps in user folders.


Were there Apple apps in those folders? Any pattern to what was skipped? I'd expect TM to back up anything it finds there.

No Apple apps. One account had Spotify only. One of the other accounts had a couple of games. No apps back up for the third account. Doesn't seem to be a pattern.


I just ran a backup of the laptop to the same TM. Issue seems to have resolved itself after a fresh install of OSX. Latest backup has Applications, Library, System and Users now. Quick check through them all seems to show that they all have the appropriate things in them. The old backups definitely didn't include any of these folders.


I'll definitely keep an eye on the backups from now on, but I'm hoping that the re-install fixed it.

Jul 11, 2013 8:19 PM in response to jmrz

jmrz wrote:

. .

Issue seems to have resolved itself after a fresh install of OSX. Latest backup has Applications, Library, System and Users now.

Yes, that got you a fresh copy of the preferences file: /Library/Preferences/com.apple.TimeMachine.plist


Some app or process, we don't know what, is changing it to exclude the top-level folders.


If it happens again, please let us know before doing anything else, so we can get you to either call AppleCare or file a Bug Report, so Apple can try to figure out what's causing this.


Thanks.

Jul 15, 2013 12:42 PM in response to Pondini

My hard drive failed, and when I went just now to restore from Time Machine, the latest complete backup is from March 12, 2013, even though the Time Machine has been plugged in and backing up this whole time. I had no idea this was an issue until I found this thread.


Is there any way I can retrieve the 4 months work that is missing, even manually? Are the missing files somewhere in Time Machine?


I had a total false sense of security since I've succesfully restored from Time Machine in the past and it looked for all intents an purposes to be doing its job. This is a disaster.


Shouldn't Apple notify people that this is an issue since there is no way to know it is occuring until it is too late, and the consequences are potentially catastrophic?

Jul 15, 2013 1:15 PM in response to miakt

miakt wrote:

. . .

Is there any way I can retrieve the 4 months work that is missing, even manually? Are the missing files somewhere in Time Machine?

Yes, you should be able to recover most of it -- especially your home folder(s).


But you have lost any apps, Internet plug-ins, and Widgets installed for all users since March 12. You've also lost any serial numbers / purchase keys for paid apps. And most "complex" apps (typically the ones that came with their own installers) have lost their support files, so may not work properly, or at all, and will have to be reinstalled from the original discs or downloaded again.


Also printer drivers, system-wide settings and configuration, and various other things in the top-level Library folder.


I had a total false sense of security since I've succesfully restored from Time Machine in the past and it looked for all intents an purposes to be doing its job. This is a disaster.

Yup. 😟


Shouldn't Apple notify people that this is an issue since there is no way to know it is occuring until it is too late, and the consequences are potentially catastrophic?

I'm not sure they've identified it as a problem; I've not had a great deal of luck getting folks to report it. (It doesn't help much once your drive has failed, as the evidence is gone.) And since I've not figured out how to replicate it, I can't report it myself.


Even when they do identify such a thing, they never warn anyone. The most they do is provide a fix via an update.



Ok, end of rant. Here's what you need to do:


First, do a full system restore from the most recent full backup in March (see Time Machine - Frequently Asked Question #14 if you're not sure how to do that).


Then you'll need to restore the home folder(s) from the most recent backup, via the TM browser (the "Star Wars" display). That's a bit of a problem, though, since you can't restore the home folder for the user account you're running from, since it's in use.


So, if you have two user accounts, log on to one. If you don't have a second user account, create one via System Prefs > Users & Groups, log off your normal account and onto the new one.

Either select your internal HD in a Finder sidebar, or press Shift+Cmd+C to show your Computer, then select your internal HD from the center part of the screen.


Under your internal HD should be a Users folder; inside that should be the home folders for each of your User Accounts (plus the special Shared folder), something like this (with the Finder in List View):


User uploaded file


Select the home folder for the other account (the one you're not logged-on to, click the Restore button at the lower-right, authenticate, and when you get a prompt about how there's already a folder there, select Replace.


Then log on to the restored user account and make sure everything's there. Repeat for the other home folder.


With the exceptions noted above, that should get you mostly recovered.


Don't hesitate to post back if you have questions or problems, and let us know how it goes.


Message was edited by: Pondini

Jul 15, 2013 2:16 PM in response to miakt

There aren't really any error logs (but most of your logs are gone anyway, since they're not backed-up).


What's happening is, some process (OSX or 3rd-party, we don't know) is putting specific exclusions in the Time Machine preferences file to exclude the top-level Applications, Library, System, and some hidden folders. That file is in Library, so is now gone. The way it's done cannot be done via the TM Preferences window, and aren't shown there, either, so the only way you know things aren't getting backed-up is to look at the top level of your HD in your backups.


If you're covered by AppleCare and want to rattle them (800-275-2273 in the U.S), by all means do so. They won't have much to work with, but don't hesitate to refer them to this thread (there are others) and/or my web page: http://pondini.org/TM/D10.html



I have the evidence (copies of the file) from a number of users, and would be glad to speak with Apple about it, either by phone or e-mail, but since I cannot replicate the problem, and am not covered by AppleCare any more myself, they're unlikely to want to do that.

Jul 21, 2013 2:25 PM in response to Pondini

Well, I have spent pretty much every waking moment since my last post trying to restore my HD. It's been a nightmare. I tried restoring my most recent back-up from Time Machine, and it kept stopping at .7% and saying "An error occured while restoring from the backup drive." I ran disk repair on the TM drive and volume and it came back clean. I tried different recovery disks, including the Snow Leopard install disk, which actually got to 3%. I tried running Migration Asst., which could not detect the TM drive at all, even though I can see it on the desktop and go through all the folders. I tried reinstalling Mountain Lion and running Setup Assistant, and it also could not find the drive.


The only thing I have been able to do is install ML on the new HD, open TM, and restore my most recent User folder from there (which was actually from 10 minutes before the failure). Of course it would not allow me to restore the System, Applications, or Library folders, so I had to go through and pick out individual non-Apple files and restore those. Things seem to be working ok for the time being.


With Recovery software, I was able to find the TM preference files from later than the most complete back-up, and indeed, System, Library and Apps had been excluded from backup after March 12. I can send you this file, Pondini if you think it would be helpful. Is it possibly related to the ML update that came out that month, since several users seem to date the problem from around then?


My main issues now are


1. When my hard drive failed, it had nearly 800GB of data on it. The new one has less than 500GB. I can't figure out what I'm missing. I compared the sizes of the current and backed up Sys, Lib, and Apps folders and the difference is only a couple of GB. There are also a lot of photos because I didn't back up iPhoto, but I don't think it's 300GB worth.


2. What am I supposed to do about Time Machine now? It seems my back-up is corrupt as well as incomplete. How do I know if it's the folder or the external drive it is on that is damaged? Should I make a new complete back up to this drive? Should I delete the old back-ups? It seems I can still pull anything off of there that the Mac isn't currently using. In the future, how can I tell if the back-up is corrupt before it is too late? I've used TM in the past to do a full restore and it worked perfectly, so now I don't know if I can trust it at all.

Jul 21, 2013 2:48 PM in response to miakt

miakt wrote:

. . .

With Recovery software, I was able to find the TM preference files from later than the most complete back-up, and indeed, System, Library and Apps had been excluded from backup after March 12. I can send you this file, Pondini if you think it would be helpful.

Won't hurt (send to the address on my Contact page and please identify yourself if your e-mail address isn't a clue), but the main place it needs to go is Apple. If you're covered by AppleCare, please give them a call as in my previous post. Not much they can do now, but the more reports they get, with that preference file, the better the chances they'll recognize the problem and perhaps find the cause.


Is it possibly related to the ML update that came out that month, since several users seem to date the problem from around then?

It doesn't look like it -- some of the failures seem to have happened last November. But we stil have no clue what's triggering it.



1. When my hard drive failed, it had nearly 800GB of data on it. The new one has less than 500GB. I can't figure out what I'm missing. I compared the sizes of the current and backed up Sys, Lib, and Apps folders and the difference is only a couple of GB. There are also a lot of photos because I didn't back up iPhoto, but I don't think it's 300GB worth.

It's possible that was system work files, caches, logs, trash, etc., in addition to your photos. That's usually only a few GBs, but can be quite large in some circumstances. No way to tell now, of course. 😟



2. What am I supposed to do about Time Machine now? It seems my back-up is corrupt as well as incomplete. How do I know if it's the folder or the external drive it is on that is damaged? Should I make a new complete back up to this drive? Should I delete the old back-ups? It seems I can still pull anything off of there that the Mac isn't currently using. In the future, how can I tell if the back-up is corrupt before it is too late? I've used TM in the past to do a full restore and it worked perfectly, so now I don't know if I can trust it at all.

I'd recommend a couple of things:


• Start making "secondary" backups to a different drive, with a different app. Just as no hardware works forever, no app is perfect, either. That's always prudent, but even more so if there's the slightest doubt about your primary backups. See the green box in Time Machine - Frequently Asked Question #27 for some suggestions.


• If you're sure you don't need anything from the (partial) old backups, your best bet might be to reformat the TM drive (perhaps taking the time to zero-it out) and start fresh, just to eliminate anything that might be left over -- either a data or directory problem Disk Utility couldn't find, and reduce the possibility there's a hardware problem (if the zero-out fails, you know there's a problem; if it succeeds, the drive is likely ok, but there's never any certainty). Then periodically check your backups, by verifying that the Applications, Library, System, etc., folders are there, until Apple provides a fix.

UPDATED: Serious Time Machine bug on Mountain Lion

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple ID.