Conflict between Time Machine and Sugarsync

See if the affected folders are listed either in Time Machine Preferences > Options.

If not, try the command in the yellow box of #11 in Time Machine - Frequently Asked Questions.

Have you tried the command in the yellow box of #11 in Time Machine - Frequently Asked Questions?

If so, with what result? If not, why not?

Ok, so that rules that out.

It's a long shot, but try a "full reset" of Time Machine, per #A4 in Time Machine - Troubleshooting. If that file is damaged, all sorts of odd things can happen.

I found two similar threads on SugarSync's forum: http://sugarsync.hivelive.com/posts/5c97e96dc8

and http://sugarsync.hivelive.com/posts/8933746d38

Looks like some of those posts may be yours, and one was from before Lion was released. There doesn't seem to be any info coming from SugarSync -- the first one says the original poster has been contacted, but there's no further information.

Other than the extended attribute you ruled out, I can't imagine why Time Machine would fail to back up those folders, although on occasion folders do get damaged in some way and Time Machine either backs-up everything in them every time, or nothing. Exactly what's wrong is not clear.

Is this consistent? Meaning, once a folder is affected, it stays affected 100%? If so, try renaming the folder, creating a new one (but not by copying or duplicating it), then move the contents into the new one (you'll probably have to re-enter it in SugarSync). The next backup should back up everything, of course, but see what happens thereafter.

Another possible fix is to force a "deep scan." Start up from your Recovery HD and Repair your internal HD, then reboot normally. Use the widget in #A1 of Time Machine - Troubleshooting to display the backup messages from your logs. The next backup should take somewhat longer than usual, in the "Preparing" phase, and there should be a "deep scan" message in the log. See what happens thereafter.

superfreud wrote:

. . .

I have already done a full Time Machine reset on all affected Macs as described in #A4 and things seemed to work for a day but then I started noticing the same behavior again (and no error message from Time Machine).

That's a clue. 😉 See if you can verify that it's completely consistent, on at least one Mac.

If it is, it sounds like SugarSync is somehow altering the plist file (that may or may not show up when you go to Time Machine Prefs > Options).

What we'll need to do to confirm it is, look at the prefs file with a different app.

I had once contacted them for the same issue but we ended up concluding that my problem was due to using the verify disk command on 10.5.x (which I see that you are mentioning on your website).

No, that applied to 1.6.0 through 10.6.7 only.

Post back with your results, then we'll take a look at the plist file.

superfreud wrote:

. . .

Forgive me if that sounds naive, but rather than SugarSync messing up with the backup exclusions, wouldn't it be possible that files saved or updated by SugarSync itself wouldn't be "recognized" by the Finder as modified in the affected folders hence preventing said files from being backed up by Time Machine.

If so, it would be a first. Time Machine normally uses the File System Event Store, a hidden log of changes that OSX keeps on each disk/partition, to figure out what's changed since the last backup.

The thing is that it seems that affected folders are those where modifications have occured through SugarSync and not "manually". By manually, I mean creating or modifying a file in the folder directly.

Yes, I understand. But all file modifications are made by OSX, so should be reflected in the FSEvents log.

Not sure I'm making myself clear. Are you familiar with SugarSync?

I don't use it, but looked at their documentation briefly.

That doesn't make a lot of sense to me, either.

I can see that doing a full restore would get SugarSync's database on your Mac out of sync with what's on their servers. You might have gotten that answer because it may be the only Time Machine related problem they're familiar with.

I can think of only 3 possible ways some files wouldn't get backed-up by Time Machine:

• Putting the extended attribute on some items (which you seem to have ruled out)
• Putting them into the Time Machine plist that stores the exclusions (and in such a way that doesn't show up via TM Prefs > Options).
• Damaging/corrupting the folder in question. We do see that on rare occasions, but don't have a clue as to exactly what happens, much less how. The only fix is to create a new folder and move the contents.

I'm not sure whether the problem affects everything in a particular folder -- once the problem starts on a folder, does that stop all backups of all items inside it, or is it just individual files?

You can use one of the apps in #A2 of Time Machine - Troubleshooting to see exactly what's getting backed-up each time.

One thing that might be worth trying is to identify a file that was added/updated at a particular time in the last 24 hours (so the backup won't be deleted) then look at the next backup to be sure it wasn't backed-up.

If it wasn't, we can check the hidden log Time Machine makes in each backup. That's a bit tedious, however.

The other thing we can try is to look at the prefs file with a 3rd-party app to see if the file is listed there. Also a bit tedious.

Let me know if you want to try one or both of these.

superfreud wrote:

It looks like when the problem starts on a folder, that it stops all backups of all items inside it. However, sometimes, creating a new file inside the aforementioned folder will re-enable a correct TM backup of that folder.

That's interesting, and may provide a clue, although it doesn't seem to match the way OSX works.

For an overview of the File System Event Database, see the blue box in How Time Machine works its Magic.

And yes, many processes use it -- Spotlight, Time Machine, some anti-virus apps (yechh!), really anything that needs to know what's new or changed lately.

But there should be no way a process can tamper with it -- as I understand it, all a process can do is request changes by event number, then do whatever's necessary and store the last event number processed, so it can start there the next time.

Therefore, when SugarSync detects a change on one Mac, and sends a copy of that file to the other one, it doesn't need anything from the EventStore on the "target" Mac -- it should just write the file to the proper folder.

It does make me wonder if SugarSync is damaging the folder somehow, in a way that makes Time Machine ignore it, until you create a file normally and fix whatever was wrong.

Very curious. 😕

superfreud wrote:

It looks like SugarSync has finally acknowledged this issue and detailed a workaround. It is not the most elegant workaround but at least I feel reassured that there is a real issue and that my testing was not for nothing.

Great! But it doesn't address how their software manages to do this (still a mystery to me).

Even stranger is this line: "If you're using Time Machine or the Verify Disk utility to save backups of your Mac, . . ." Using Verify Disk to "save backups"? Huh?

All the workaround does is guarantee everything in the folder will be backed-up next time (regardless of whatever their software is doing, and regardless of whether it was backed-up properly before). Better than nothing, I guess . . .

From your response to them:

I have more than 15 "highest-hierarchy" folders synced to Sugarsync... Excluding them on all my macs from TM backups and re-adding them on a regular basis to make sure that everything is backed up properly does not seem like a viable option to me.

If you're familiar with Applscript, UNIX, and Terminal, there is a way to automate that, if you want to tackle it. You'd want it to run, say, once per day. The new tmutil command allows setting and removing exclusions, and running a backup. A complication is, the verbs require root privileges.

I am glad you raised this issue. I'm still using Apple's old Backup app (part of a MobileMe subscription) to back up some things to my iDisk daily. That isn't going to work with iCloud, and MobileMe will evaporate on June 30, so I'm going to have to do something else. From a quick review, DropBox and SugarSync both looked like reasonable alternatives. Maybe not.

If you hear anything more, let us know.