Previous 1 2 Next 17 Replies Latest reply: Jan 12, 2012 11:29 PM by superfreud
superfreud Level 1 Level 1 (5 points)

Hi everyone,

 

I was checking the integrity of my Time Machine backups for my differents macs and realized that some folders synced to SugarSync aren't backep up properly by Time Machine.

I've been able to replicate this problem on 4 different macs running 10.7.2 and the latest build of Sugarsync for Mac.

 

What happens is:

Initial Time Machine backup works fine. Incremental time machine backups work for some time and after a while changes in synced folders aren't updated in Time Machine backups although they are replicated in Sugarsync and amongst my computers through Sugarsync (but not on the Time Machine backups of the computers).

This is happening on my 4 Macs. I have another Mac where Sugarsync isn't installed which is not affected by this problem.

The only common factor between all these macs is Sugarsync.

 

As this problem is not easy to replicate, I invite all users using Time Machine to check their backups for missing files.

 

My main concern is that Time Machine seems to be running well, there are no error messages but it does not back up some files as if it did not "see" the changes that occured in synced folders.

 

I've already tried to reset the time machine backups and restart from scratch but this problem keeps re-occuring.


iMac, Mac OS X (10.7.2), i7 3,4 Ghz, 16 Gb RAM, 2 Tb HD
  • Pondini Level 8 Level 8 (38,715 points)

    See if the affected folders are listed either in Time Machine Preferences > Options.

     

    If not, try the command in the yellow box of #11 in Time Machine - Frequently Asked Questions.

  • superfreud Level 1 Level 1 (5 points)

    Thanks for your answer.

     

    I haven't set up any exclusions. It seems Time Machine fails to identify some changes in folders that are synced to Sugarsync when the changes are made from a different computer and "replicated" on the affected computer.

  • Pondini Level 8 Level 8 (38,715 points)

    Have you tried the command in the yellow box of #11 in Time Machine - Frequently Asked Questions?

     

    If so, with what result?  If not, why not?

  • superfreud Level 1 Level 1 (5 points)

    Here's the result from one of the affected macs:

     

    Users/superfreud/Library/Calendars/Calendar Cache

    /Users/superfreud/Library/Saved Application State

    /Users/superfreud/Library/Safari/WebpageIcons.db

    /Users/superfreud/Music/iTunes/iTunes Music Library.xml

    /Users/superfreud/Pictures/iPhoto Library/iPod Photo Cache

    /Users/superfreud/Pictures/iPhoto Library/AlbumData.xml

    /Users/superfreud/Music/iTunes/Album Artwork/Cache

    /Users/superfreud/Library/iTunes/iPad Software Updates

    /Users/superfreud/Library/Containers/com.apple.TextEdit/Data/Library/Caches

    /Users/superfreud/Library/Application Support/Google/Chrome/Default/Favicons

    /Users/superfreud/Library/Application Support/Google/Chrome/Default/History

    /Users/superfreud/Library/Application Support/Google/Chrome/Safe Browsing Csd Whitelist

    /Users/superfreud/Library/Application Support/Google/Chrome/Safe Browsing Bloom

    /Users/superfreud/Library/Application Support/Google/Chrome/Safe Browsing Bloom Filter 2

    /Users/superfreud/Library/Application Support/Google/Chrome/Safe Browsing Download

    /Users/superfreud/Library/Containers/com.apple.Preview/Data/Library/Caches

     

     

    None of the affected folders are listed above.

  • Pondini Level 8 Level 8 (38,715 points)

    Ok, so that rules that out.

     

    It's a long shot, but try a "full reset" of Time Machine, per #A4 in Time Machine - Troubleshooting.  If that file is damaged, all sorts of odd things can happen.

     

    I found two similar threads on SugarSync's forum:  http://sugarsync.hivelive.com/posts/5c97e96dc8

    and http://sugarsync.hivelive.com/posts/8933746d38

     

    Looks like some of those posts may be yours, and one was from before Lion was released.  There doesn't seem to be any info coming from SugarSync -- the first one says the original poster has been contacted, but there's no further information.

     

    Other than the extended attribute you ruled out, I can't imagine why Time Machine would fail to back up those folders, although on occasion folders do get damaged in some way and Time Machine either backs-up everything in them every time, or nothing. Exactly what's wrong is not clear.

     

    Is this consistent?  Meaning, once a folder is affected, it stays affected 100%?  If so, try renaming the folder, creating a new one (but not by copying or duplicating it), then move the contents into the new one (you'll probably have to re-enter it in SugarSync).  The next backup should back up everything, of course, but see what happens thereafter.

     

    Another possible fix is to force a "deep scan."  Start up from your Recovery HD and Repair your internal HD, then reboot normally.  Use the widget in #A1 of  Time Machine - Troubleshooting to display the backup messages from your logs. The next backup should take somewhat longer than usual, in the "Preparing" phase, and there should be a "deep scan" message in the log.  See what happens thereafter.

  • superfreud Level 1 Level 1 (5 points)

    I have this issue on 4 different iMacs with 4 different installations backing up on either a Time capsule (I have 2 Time Capsules, same problem for both) or an external HD.

     

    I can't fathom that all 4 iMacs or backup HD (TC or external) would all be affected by the exact same issue.

     

    As for the consistency question: it is not consistent as some folders won't show the behavior described above and those that show it might be backed up correctly after a while (usually when a change is made on the local machine inside the folder affected by this problem but not 100% of the time).

     

    The only thing that I know for sure is that I can't trust Time Machine for my backup strategy.

     

    I have already done a full Time Machine reset on all affected Macs as described in #A4 and things seemed to work for a day but then I started noticing the same behavior again (and no error message from Time Machine).

     

    The only Mac that I have that is not affected by this problem is not using Sugarsync so I'm tempted to think Sugarsync plays a role there by the way it handles file syncronization.

     

    I have contacted SugarSync support but they're usually very slow to react so we'll see.

     

    I had once contacted them for the same issue but we ended up concluding that my problem was due to using the verify disk command on 10.5.x (which I see that you are mentioning on your website). I'm pretty sure that the problem was the same at the time. It's just that after a full time machine reset at the time, things seem to be working after a day of testing so I ended thinking that the problem had been isolated to using the verify disk command (I haven't used that command recently so it couldn't be causing the problem). I'm thinking now that I've been backing up my computers for more than a year with Time Machine not knowing that there were files missing in my backups...

  • Pondini Level 8 Level 8 (38,715 points)

    superfreud wrote:

    . . .

    I have already done a full Time Machine reset on all affected Macs as described in #A4 and things seemed to work for a day but then I started noticing the same behavior again (and no error message from Time Machine).

    That's a clue. See if you can verify that it's completely consistent, on at least one Mac. 

     

    If it is, it sounds like SugarSync is somehow altering the plist file  (that may or may not show up when you go to Time Machine Prefs > Options).

     

    What we'll need to do to confirm it is, look at the prefs file with a different app. 

     

    I had once contacted them for the same issue but we ended up concluding that my problem was due to using the verify disk command on 10.5.x (which I see that you are mentioning on your website).

    No, that applied to 1.6.0 through 10.6.7 only.

     

    Post back with your results, then we'll take a look at the plist file.

  • superfreud Level 1 Level 1 (5 points)

    I had once contacted them for the same issue but we ended up concluding that my problem was due to using the verify disk command on 10.5.x (which I see that you are mentioning on your website).

    No, that applied to 1.6.0 through 10.6.7 only.

     

    Yes, I meant 10.6.x sorry about that.

     

     

    Forgive me if that sounds naive, but rather than SugarSync messing up with the backup exclusions, wouldn't it be possible that files saved or updated by SugarSync itself wouldn't be "recognized" by the Finder as modified in the affected folders hence preventing said files from being backed up by Time Machine.

     

    (I don't know exactly how TM works so it's only a naive assumption).

     

    The thing is that it seems that affected folders are those where modifications have occured through SugarSync and not "manually". By manually, I mean creating or modifying a file in the folder directly.

     

    Not sure I'm making myself clear. Are you familiar with SugarSync?

     

    (Once again thanks again for your help).

     

    I'll check the plist file on the affected computers later on today.

  • Pondini Level 8 Level 8 (38,715 points)

    superfreud wrote:

    . . .

    Forgive me if that sounds naive, but rather than SugarSync messing up with the backup exclusions, wouldn't it be possible that files saved or updated by SugarSync itself wouldn't be "recognized" by the Finder as modified in the affected folders hence preventing said files from being backed up by Time Machine.

    If so, it would be a first.  Time Machine normally uses the File System Event Store, a hidden log of changes that OSX keeps on each disk/partition, to figure out what's changed since the last backup.

     

     

    The thing is that it seems that affected folders are those where modifications have occured through SugarSync and not "manually". By manually, I mean creating or modifying a file in the folder directly.

    Yes, I understand. But all file modifications are made by OSX, so should be reflected in the FSEvents log.

     

    Not sure I'm making myself clear. Are you familiar with SugarSync?

    I don't use it, but looked at their documentation briefly.

  • superfreud Level 1 Level 1 (5 points)

    SugarSync support did answer to my post on their website:

     

    https://sugarsync.custhelp.com/app/account/questions/detail/i_id/279999

     

    They seem to acknoledge that SugarSync can be responsible for Time Machine misbehavior but they did not address my issue. Maybe it will make more sense to you...

     

    Thanks again for your persistent help!

  • Pondini Level 8 Level 8 (38,715 points)

    That doesn't make a lot of sense to me, either.

     

    I can see that doing a full restore would get SugarSync's database on your Mac out of sync with what's on their servers.  You might have gotten that answer because it may be the only Time Machine related problem they're familiar with.

     

    I can think of only 3 possible ways some files wouldn't get backed-up by Time Machine:

     

    • Putting the extended attribute on some items (which you seem to have ruled out)
    • Putting them into the Time Machine plist that stores the exclusions (and in such a way that doesn't show up via TM Prefs > Options).
    • Damaging/corrupting the folder in question.  We do see that on rare occasions, but don't have a clue as to exactly what happens, much less how.  The only fix is to create a new folder and move the contents.

     

    I'm not sure whether the problem affects everything in a particular folder -- once the problem starts on a folder, does that stop all backups of all items inside it, or is it just individual files?

     

    You can use one of the apps in #A2 of Time Machine - Troubleshooting to see exactly what's getting backed-up each time.

     

    One thing that might be worth trying is to identify a file that was added/updated at a particular time in the last 24 hours (so the backup won't be deleted) then look at the next backup to be sure it wasn't backed-up.  

    If it wasn't, we can check the hidden log Time Machine makes in each backup.  That's a bit tedious, however.

     

    The other thing we can try is to look at the prefs file with a 3rd-party app to see if the file is listed there.  Also a bit tedious.

     

    Let me know if you want to try one or both of these. 

  • superfreud Level 1 Level 1 (5 points)

    It looks like when the problem starts on a folder, that  it stops all backups of all items inside it. However, sometimes, creating a new file inside the aforementioned folder will re-enable a correct TM backup of that folder.

     

    I'm wondering if by any chance SugarSync isn't using TM (OS X actually)'s method of detecting changes in folders in order to update its own backups to SugarSync servers and synced computers. It would make sense that it might, in some cases, modify system files and that TM wouldn't be able to pick up the changes afterwards (again that's my "naive" way of seeing things... I have no insight as to how exactly TM or Sugarsync work, nor am I computer-savvy enough to claim to understand this).

     

    The only thing that I can think of is that SugarSync and TM share similar functionalities: i.e. making incremental backups of folders according to changes detected in these folders. I believe it sounds logical that they might use the same common files in order to detect changes and maybe that is what's screwing things up for TM... Once again this is my layman's view...

  • Pondini Level 8 Level 8 (38,715 points)

    superfreud wrote:

     

    It looks like when the problem starts on a folder, that  it stops all backups of all items inside it. However, sometimes, creating a new file inside the aforementioned folder will re-enable a correct TM backup of that folder.

    That's interesting, and may provide a clue, although it doesn't seem to match the way OSX works.

     

    For an overview of the File System Event Database, see the blue box in How Time Machine works its Magic.

     

    And yes, many processes use it -- Spotlight, Time Machine, some anti-virus apps (yechh!), really anything that needs to know what's new or changed lately.

     

    But there should be no way a process can tamper with it -- as I understand it, all a process can do is request changes by event number, then do whatever's necessary and store the last event number processed, so it can start there the next time.

     

    Therefore, when SugarSync detects a change on one Mac, and sends a copy of that file to the other one, it doesn't need anything from the EventStore on the "target" Mac -- it should just write the file to the proper folder.

     

    It does make me wonder if SugarSync is damaging the folder somehow, in a way that makes Time Machine ignore it, until you create a file normally and fix whatever was wrong.

     

    Very curious. 

  • superfreud Level 1 Level 1 (5 points)

    It looks like SugarSync has finally acknowledged this issue and detailed a workaround. It is not the most elegant workaround but at least I feel reassured that there is a real issue and that my testing was not for nothing.

     

    I do hope that they will address this bug in a future release of their software.

     

    Thanks again Pondini for your help and for taking the time to try and figure things out.

     

    Here are the links on SugarSync website:

     

    https://sugarsync.custhelp.com/app/answers/detail/a_id/539/

     

    http://sugarsync.hivelive.com/posts/5c97e96dc8

Previous 1 2 Next