13 Replies Latest reply: Jan 4, 2011 1:11 AM by erydrabl
Geoff1234 Level 1 Level 1 (0 points)
I've run into a pretty serious problem with Time Machine, and I wonder if anybody can figure out what is going wrong. Recently, Time Machine has stopped backing up changed files. It runs every hour as expected without any error messages or warnings, and when I run Time Machine to look at the backups, the hourly backups are all available. However, when I examine the backup files themselves, there have been no updates to the files since December 9th. Many files have been changed since Dec. 9. I manually checked a few of the files that have been changed since Dec. 9, and they were indeed the file as of Dec 9, so it is definitely not the correct file with a mislabeled modification time.
To me, this is a rather serious problem, since in my opinion it is worse to think you have backups when you don't than to have no backups at all.
This is what I've done to try to solve this, all without any success.
The backup is being done to a Time Capsule. I rebooted the Time Capsule with power off (not reset), and rebooted my Mac.
My daughter is also backing up her MacBook (also running Snow Leopard) to the same Time Capsule. Her backups are working properly. This leads me to believe that there is nothing wrong with the Time Capsule. We are both connecting to the Time Capsule using the Air Port, and the success of her backups also indicates that the network connection is OK. I am also able to save unrelated files to the Time Capsule from my computer, as further confirmation that the disk and air port are working properly.
I checked the disk image (the sparsebundle file) using Disk Utility. No problems were found.
I changed the name of the sparsebundle file on the Time Capsule, and recreated the backup. This worked. I did this on Dec 12, and the files in the new backup were up to date as of Dec 12. However, nothing subsequent to Dec 12 was updated in the new backup.
I attached a USB external drive to Time Capsule using the USB port on the Time Capsule. I changed the backup drive to the new drive, and created a new Time Machine backup on the new drive. That process worked; however, subsequent Time Machine updates to this drive did not update any changed files.
I cleared out the spotlight cache with the command sudo mdutil -E / . This appeared to clear the spotlight cache successfully, since Spotlight was unable to find anything immediately after running the command, and mds immediately started working to reindex the disk. However, this did not solve the problem.
All of this indicates to me that there is something wrong on my MacBook. Since rebooting did not solve the problem, the problem is probably with a file. But that is just my conclusion based on the observed behavior. I am not an expert in how Time Machine works, but perhaps somebody who knows more than I can point out some other things that I can try.
It also seems to me that Apple should like to know about this problem, because in my mind it raises a serious question about the integrity of the Time Machine backup process.
Thanks in advance for your help. I'd like to find a solution to this that does not involve reinstalling the operating system or other draconian measures. Those sort of solutions strike me as PCish.

MacBook Pro, Mac OS X (10.6.5)
  • 1. Re: Time Machine fails to backup without warning or error message
    Pondini Level 8 Level 8 (38,720 points)
    Hi, and welcome to the forums.

    A clue may be lurking in your logs. See #A1 in [Time Machine - Troubleshooting|http://web.me.com/pondini/Time_Machine/Troubleshooting.html] (or use the link in *User Tips* at the top of this forum).

    Copy and post the messages for one of these backups here.
  • 2. Re: Time Machine fails to backup without warning or error message
    Geoff1234 Level 1 Level 1 (0 points)
    Further efforts. (Sorry in advance for being so verbose, but there may be a useful clue in what I've tried so far.)

    I discovered that the .Spotlight-V100 file in the / directory of my Mac hard disk had a Dec 9th timestamp, in spite of running the command mdutil -E /. Dec 9th was the day the Time Machine backups stopped working. Therefore, I deleted the directory entirely using sudo rm -rf /.Spotlight-V100. Spotlight rebuilt the index after the original index was erased. Spotlight seemed to be functioning properly, but Time Machine update were still not working.

    I downloaded the Time Machine Buddy widget and looked at the logs. There were no indications of any problems in the logs, and the steps taken in the backup, as indicated in the logs, were identical to the steps taken in the successful backups on my daughter's computer.

    I looked at some of the tips in the troubleshooting website. I did a full reset of Time Machine (A4). The reset was successful, but the update failure persisted.



    I followed the steps in tip D5. I got down to the last step without any success. (Including trying Restart to solve the problem mentioned in the yellow box on that page.) The last step was:

    . "If still no help, create (or duplicate) some "test" files in various places in your home folder, another user's home folder (if you only have one, make another temporarily via System Preferences > Accounts), and one in your Applications folder and/or at the top level of your internal HD.  Then run a Back Up Now from the Time Machine icon in your Menubar, or by right-clicking the Time Machine icon in your Dock.  If some get saved, but not others, something is likely corrupted.  Post a new thread in the appropriate Apple Discussions forum with full details, including which files were and were not backed-up, and someone will help you fix it.
"

    I created some test files, named TimeMachineTest, in various directories using the command sudo touch TimeMachineTest. I created the files in these directories:
    /System
    /Applications
    /Users/<user1>/Documents
    /Users/<user2>/Documents
    /Users/<user3>/Documents

    where <user1> is me, and <user2> and <user3> are two other user accounts. After creating these files, I ran the Time Machine Backup. The files in the two system folders were successfully backed up, while the files in /User directories were not. This was at least some progress.

    I did some further reading on the internet, and somebody suggested that there was something wrong in the user accounts, and that I could solve the problem by creating a new user account for myself and moving all of my files to the new directory. This seemed like a possibility, so I created the new account. Before I moved any files, I tried running a backup. This backup was successful. 58 GB of data was transferred to the Time Machine disk image, including all of the TimeMachineTest files in the users' directories. FWIW, the size of my /Users folder is about 55GB, so it seems possible that there is an instruction for Time Machine to back up the entire /Users folder whenever a new account is added, or an existing account deleted, and that this instruction is taking precedence over whatever is causing my problem.

    I then created new test files named TimeMachineTest1 in each of the directories, and tried the backup again. This backup was unsuccessful - none of the "TimeMachineTest1" files were backed up. (I'd just like to reiterate at this point that by saying "unsuccessful", I do not mean that there were errors or warnings. Every backup that I have run has been nominally successful. The backups have only been "unsuccessful" in a functional sense, i.e. the data was not actually backed up.)

    I tried backing up from the new user account with no success. This indicated to me that moving my files to the new user account would not accomplish anything, so I gave up on that idea. I deleted the new user account. After deleting the account, the backup was again successful. Another 58 GB of data was transferred to the Time Machine disk image, including all of the "TimeMachineTest1" files.

    Since the files in the /User directory were not being backed up, perhaps there is some sort of exclusion on that directory. (Although if there were an exclusion, then perhaps the /User directory would not be backed up when the new user account was created or deleted.) There were no exclusions in the Time Machine Preferences GUI. I checked the file /System/Library/CoreServices/backupd.bundle/Contents/Resources/StdExclusions.pl ist, which had nothing suspicious in it.

    I cleaned out all of the cache files that I could find, but the backups still did not work.

    I ran "Repair Permissions" on my "Macintosh HD". There were many permission problems, but repairing them did not solve the problem.

    I should also mention that I had done "Shutdown" several times throughout this process.

    At this point, I'm not sure what to try. According to the instructions on the troubleshooting website, "something is likely corrupted". This isn't a very satisfying answer, but if it is the case, what do I do about it?
  • 3. Re: Time Machine fails to backup without warning or error message
    Pondini Level 8 Level 8 (38,720 points)
    Geoff1234 wrote:
    . . .
    I did some further reading on the internet, and somebody suggested that there was something wrong in the user accounts, and that I could solve the problem by creating a new user account for myself and moving all of my files to the new directory. This seemed like a possibility, so I created the new account. Before I moved any files, I tried running a backup. This backup was successful. 58 GB of data was transferred to the Time Machine disk image, including all of the TimeMachineTest files in the users' directories. FWIW, the size of my /Users folder is about 55GB, so it seems possible that there is an instruction for Time Machine to back up the entire /Users folder whenever a new account is added, or an existing account deleted, and that this instruction is taking precedence over whatever is causing my problem.


    No, there's no such instruction; and what you're seeing is quite unusual.

    We do see an occasional home folder that behaves that way; it only gets backed-up on an initial backup (TM uses different programming for that).

    But it appears that something is wrong with your /Users folder. Being intermittent, though, is also unusual; I don't recall ever seeing it before.

    Before tackling that, though, let's rule out a problem with your installation of OSX. I'd suggest downloading and installing the 10.6.5 "combo" update. That's the cleverly-named combination of all the updates to Snow Leopard since it was first released, so installing it should fix anything that's gone wrong since then, such as with one of the normal "point" updates. Info and download available at: http://support.apple.com/kb/DL1324 Be sure to do a +Repair Permissions+ via Disk Utility (in your Applications/Utilities folder) afterwards.

    If that doesn't help, reinstall OSX from your Snow Leopard Install disc (that won't affect anything else), then apply the "combo" again.
  • 4. Re: Time Machine fails to backup without warning or error message
    Geoff1234 Level 1 Level 1 (0 points)
    I installed the combo update, and repaired disk permissions. This did not help.

    I then installed the Snow Leopard operating system from the disk, installed the combo update from the downloaded file, and repaired disk permissions. After this, Time Machine did what appeared to be a full update, including the files in the Users directory.
    After that I created a new file in my ~/Documents folder called TimeMachineTest4. This file was not backed up by running "Back Up Now", nor by subsequent hourly updates. (The logs for all of the backup attempts indicate that they succeeded.) Therefore, it appears that a reinstall of the OS did not fix the problem.

    As an experiment, I created a new folder at the top level of the Macintosh HD called Userdata. I changed the permissions and ownership to match those of the Users folder. I created a subfolder under Userdata called "userx", with equivalent ownership and permissions to the userx subfolder under Users. I then copied the contents of /Users/userx to /Userdata/userx, including hidden files and folders.
    I created a file called TimeMachineTest5 in both /Users/userx/Documents and /Userdata/userx/Documents. The Time Machine backup was successful for the file in the Userdata folder, but not for the Users folder.

    From a unix perspective, it seems that if I were to erase the /Users/userx folder and replace it with a symbolic link to /Userdata/userx, I would have the same functionality as with the current setup, and it appears that /Userdata/userx would be backed up. However, I don't know whether the OS X GUI or the account management system would like this solution. It is a kludgy solution, but it seems no less random than reinstalling the operating system, etc.

    One thing that I haven't tried is recreating the Time Machine sparsebundle from scratch using the reinstalled OS. I'm hesitating to do this because my Time Capsule is running out of space with all of the extra backup files that I'm saving, so I'd have to copy some of them to a different disk before proceeding. Other than what happened when I added a new user account, the system is certainly behaving like there is an exclusion specified for the /Users folder somewhere, and if that is the case recreating the backup file won't accomplish anything.

    So far I've found three places where it would be possible to exclude /Users.

    1) The Options box under Time Machine Preferences. Nothing is being excluded there.
    2) The file /System/Library/CoreServices/backupd.bundle/Contents/Resources/StdExclusions.pl ist. My file appears to have identical contents to other examples that I've found, and it certainly does not have /Users listed.
    3) I have found one other file with exclusions, but I can't find the name of the file now. When I examined the file, it primarily had Xcode build directories as items to be excluded. It did not have the /Users directory, although I'm working from memory since the name of this file doesn't seem to be widely disseminated on the internet, and I must have discovered it in the first place mostly by luck.
  • 5. Re: Time Machine fails to backup without warning or error message
    Pondini Level 8 Level 8 (38,720 points)
    Geoff1234 wrote:
    . . .
    I created a file called TimeMachineTest5 in both /Users/userx/Documents and /Userdata/userx/Documents. The Time Machine backup was successful for the file in the Userdata folder, but not for the Users folder.


    Yes, that's what I was afraid of. We've seen an occasional home folder that was damaged, causing similar problems (but consistent, not intermittent like you're seeing), but I don't recall seeing a problem with the whole /Users folder.

    From a unix perspective, it seems that if I were to erase the /Users/userx folder and replace it with a symbolic link to /Userdata/userx, I would have the same functionality as with the current setup, and it appears that /Userdata/userx would be backed up. However, I don't know whether the OS X GUI or the account management system would like this solution. It is a kludgy solution, but it seems no less random than reinstalling the operating system, etc.


    Another option might be to simply move all the contents of the /Users folder to the new one, delete the old one, and rename the new one to /Users. That's a bit scary; I don't know if there are other implications of that, and I doubt you can do that while running from that volume. (I wouldn't suggest that to most users, but you seem familiar with permissions, UNIX, and Terminal.)

    One thing that I haven't tried is recreating the Time Machine sparsebundle from scratch using the reinstalled OS.


    Do you have an external HD you can try, temporarily? Backups will be much faster, of course, and it would rule out problems with the TC (which seems pretty far-fetched, but this whole problem is bizarre anyway!)

    So far I've found three places where it would be possible to exclude /Users.


    Some things are excluded via extended attributes. See #11 in [Time Machine - Frequently Asked Questions|http://web.me.com/pondini/Time_Machine/FAQ.html] (or use the link in *User Tips* at the top of this forum), and/or execute this via Terminal:

        sudo mdfind "comapple_backupexcludeItem = 'com.apple.backupd'"
  • 6. Re: Time Machine fails to backup without warning or error message
    Geoff1234 Level 1 Level 1 (0 points)
    Further update.

    I copied the TimeMachine sparsebundle file to a separate disk with enough space. I then erased the sparsebundle file from Time Capsule, and did a new Time Machine backup, using the clean OS. The first backup worked fine, but as before, subsequent backups did not work.

    I was a little nervous about moving the /Users directory around, because I don't know enough about how the OS works to fully appreciate the problems this might create. I did a unix test, and found that if you move a file away from a symbolic link and then move a second file into the place of the first one, the link still points to the file with the original name. This was encouraging for implementing your suggestion.

    I saved the /Users directory in yet another location to try to enable me to recover if something went wrong. I then copied all of /Users into a parallel directory, /Users.new, preserving permissions. I then logged in as root in single user mode, and tried to move /Users to a different location. This command failed, with the message that this was a read-only filesystem. The unix permissions for /Users should have allowed writes by root. I could have tried experimenting with permissions, but that did not seem like a wise thing to do. At least my computer works now, even if the backups don't, and I don't want to risk messing everything up.

    After this, I implemented the strategy that I suggested a couple of posts ago. I created a new directory, /Userdata, at the same level as /Users. I moved all of the folders in /Users to /Userdata, and replaced those folders with symbolic links pointing to the folders that were now in /Userdata.

    After doing this, I was still able to login, and the Mac OS Gui seems to be working properly. (E.g., clicking on the Documents icon in Finder displays the contents of /Userdata/Documents, as desired. In addition, the backups are now working properly. The Time Machine Gui even follows the links correctly. From inside the Mac OS Gui, I don't think that you can even tell that the contents of /Users are links. I don't know what is going to happen if I try to add users, but it is unlikely that I'll need to do that unless one of my dogs learns how to use the computer.

    I wouldn't say that this solves the problem, but it at least allows me to have some hope that my data will be backed up properly. I'm going to keep a close eye on Time Machine, however, since after this I'm not going to trust it.

    Thanks for your help, Pondini. Based on my searches on the internet, I would say it's possible that I'm the only person with this problem. Given that, I believe that it would be very difficult for anybody without direct access to my computer to figure out exactly what the problem is. At least I have a workable solution.
  • 7. Re: Time Machine fails to backup without warning or error message
    Pondini Level 8 Level 8 (38,720 points)
    Geoff1234 wrote:
    . . .
    Based on my searches on the internet, I would say it's possible that I'm the only person with this problem.


    I suspect so. Very odd.

    Given that, I believe that it would be very difficult for anybody without direct access to my computer to figure out exactly what the problem is.


    Yup. It does seem to be some sort of problem with the /Users folder; why it's intermittent is an even larger mystery.

    At least I have a workable solution.


    Yes, very inventive!

    It's always a good idea to keep secondary backups, and in your case I'd strongly recommend it, if you aren't already. See #27 in [Time Machine - Frequently Asked Questions|http://web.me.com/pondini/Time_Machine/FAQ.html] (or use the link in *User Tips* at the top of this forum), for some suggestions.
  • 8. Re: Time Machine fails to backup without warning or error message
    erydrabl Level 1 Level 1 (0 points)
    Geoff1234 wrote:
    Thanks for your help, Pondini. Based on my searches on the internet, I would say it's possible that I'm the only person with this problem. Given that, I believe that it would be very difficult for anybody without direct access to my computer to figure out exactly what the problem is. At least I have a workable solution.


    You are not the only person with this problem. Unfortunately I can reproduce it 100%: Everything is backed up properly expect all user directories in hourly and manual backups. I had the same issue a couple of weeks ago. Last time I had success with a full reset described here: http://web.me.com/pondini/Time_Machine/A4.html But this time it didn't work.

    For me a solution without moving and linking user directories would be highly preferred. If I can help with providing additional information just let me know.

    Thanks in advance, Adrian
  • 9. Re: Time Machine fails to backup without warning or error message
    erydrabl Level 1 Level 1 (0 points)
    As I told in my last post, my computer had exactly the same symptoms as described by Geoff1234 in this thread. But I was looking for a solution without reformatting the TM-volume or moving and linking my /Users directory. But further searches on the internet or in this forum didn't bring any help. So I decided to call Apple's paid support for assistance. That was successful. And here is what I was recommended to do:

    1. Shut down
    2. Disconnect any cables including power
    3. Remove battery
    4. Reconnect power cable and battery
    5. Reset PRAM
    6. Overwrite any cache files by moving the content of the following directories (but not the directories itself) to trash and securely empty trash
    (/Library/Caches/* /System/Library/Caches/* /Users//Library/Caches/)
    7. Reboot
    8. Reconnect FireWire drive for Time Machine

    In fact Time Machine workes again as expected.
    I hope this can help someone!

    Best regards Adrian
  • 10. Re: Time Machine fails to backup without warning or error message
    Geoff1234 Level 1 Level 1 (0 points)
    Hi Adrian,

    I was away so I didn't read your previous message until now. Thanks for tracking this down and posting your experience here. I can understand why you didn't like my solution of moving the home directory data, because I don't like it either.

    Frankly, the solution they gave you makes me kind of angry. It means that Apple knows about this problem, and the best solution they have requires you to remove the battery. I have a MacBook Pro with a sealed case, and the only way to remove the battery is to remove screws from the case. To me, that has the feel of doing something that would violate the warranty. My guess is that it might work without removing the battery, and perhaps I'll give that a try.

    Aside from that, the solution is basically saying "we don't know what the problem is, but we know that clearing out everything solves it." For something as important as data backup, that just doesn't work for me. Even after you do this, how can you ever trust the backup process again? You don't know what caused the problem, so you have no way of checking whether the problem has reoccurred, other than to manually examine the backup data frequently. At least after this experience, you and I know enough not to trust Time Machine.

    They've tried to streamline the backup process so that each incremental backup doesn't take too long, and they've been successful at that. I can do my backups over a wireless network with no problem. But what they've managed to do is create a process that has so many moving pieces that not only isn't it robust and reliable, but it is also so complicated that Apple themselves can't debug it.

    In spite of this, I still like my Mac a lot more than my PC.

    Geoff
  • 11. Re: Time Machine fails to backup without warning or error message
    hoppah Level 1 Level 1 (0 points)
    This may or may not be helpful, but I had the same symptom: Time Machine simply started and never finished, with the same message in the logs: "Waiting for index to be ready (100)". Clicking on the rotating Time Machine clock in the status bar always showed Time Machine backing up "N bytes out of X", where N was a very small number which did not change. Rebooting, starting and stopping the NAS to which it backs up, killing Time Machine and restarting it, all to no avail, and none of the remedies listed in the documentation referred to in these threads was helpful either.

    Since Time Machine had worked flawlessly for months and I knew my NAS drives to be in good working order, I racked my brains to think of what had changed. Then I remembered that I had enabled indexing of my main NAS volume with the command line mdutil option. This is supposed to enable Spotlight searching of external volumes, which is something I'd been really upset to find had been disabled in Snow Leopard. I'd also noticed that the Spotlight index had never been updated with info from my external volume but had not had the time to troubleshoot it.

    I turned off the indexing of the external volume (again with the mdutil command) and bingo, the Time Machine backup immediately progressed to finish.

    So somehow Spotlight indices are intimately tied into Time Machine's backup mechanism and if they're open or locked or otherwise broken, TM won't begin the backup. TM desperately needs a timeout mechanism on this function - having it sit for days "waiting" is most unhelpful.

    Don't know if this applies in any way to your situation, but there it is.

    G.
  • 12. Re: Time Machine fails to backup without warning or error message
    Geoff1234 Level 1 Level 1 (0 points)
    Hi,
    I don't think this is the exact same problem that I have, but I think it is related. My bet is that Spotlight is the weak link in this process, and we have each found a different way that the backup process can fail because Spotlight is not performing correctly.

    In my (and Adrian's) case, I think that Spotlight is failing to recognize that files have been changed. In our case, that makes for a quick backup, because TimeMachine thinks nothing has changed. (Or at least nothing in the Users directory has changed, which is where most of the changing data is likely to be.) In your case, Spotlight is somehow confused by trying to index external drives, can't figure out what has changed, and causes the backup to hang.

    Spotlight is an essential part of the backup process, because it is what allows backups to be incremental. My guess is that Spotlight wasn't designed for that task originally, but was co-opted for the backup task after the fact. In its original role, Spotlight didn't have to be 100% robust, because it is basically a convenience function, and failure is not fatal. However, in its role as provider of information to the backup process, it is essential that it be very robust, and we have discovered a couple of instances in which it is not up to the job.

    In your case, Spotlight is pretty clearly the culprit. In my case, it is less clear, but this is why I think it is to blame: Full initial backup, Spotlight is not consulted because all files are backed up, success. Partial incremental backup, Spotlight is consulted to determine changes, failure.

    The TimeMachine backup is not a simple process to implement. There is an intrinsic tradeoff of speed and compactness vs. accuracy. I just wish that Apple had spent more time making the backup foolproof, and less time on the fancy GUI.
  • 13. Re: Time Machine fails to backup without warning or error message
    erydrabl Level 1 Level 1 (0 points)
    Geoff1234 wrote:
    Hi Adrian,

    Frankly, the solution they gave you makes me kind of angry. It means that Apple knows about this problem, and the best solution they have requires you to remove the battery. I have a MacBook Pro with a sealed case, and the only way to remove the battery is to remove screws from the case. To me, that has the feel of doing something that would violate the warranty. My guess is that it might work without removing the battery, and perhaps I'll give that a try.


    Hi Goeff,

    I think I was not precise enough with my description. The important part of step 3 is not to remove the battery but basically to reset the the System Management Controller (SMC). Here is described how to do so on any system even for "portables with a battery you should not remove on your own".

    Geoff1234 wrote:
    Aside from that, the solution is basically saying "we don't know what the problem is, but we know that clearing out everything solves it." For something as important as data backup, that just doesn't work for me. Even after you do this, how can you ever trust the backup process again? You don't know what caused the problem, so you have no way of checking whether the problem has reoccurred, other than to manually examine the backup data frequently. At least after this experience, you and I know enough not to trust Time Machine.


    You are perfectly right. But on the one hand I'm glad that the backups seem to work again. On the other hand I feel insecure because we don't know what exactly caused the problem. So I will have a close eye on my backups. Very important data like my recent work and photographs I usually safe additionally with rsync to a second and even a third hard drive in a different location.

    Adrian