Time Machine always requires deep traversal?

Markus Altendorff wrote:

. . .

Just to add some idle speculation, maybe these old log files should be cleared out periodically by some of those maintenance scripts that the system is supposed to run every day/week/month, and something broke in there yet nobody noticed...? Then again, I've not seen these in a while, does Apple still use them?

According to this ArsTechnical article, the Event Store is not purged periodically, but that doesn't seem reasonable. It's 4 years old, though, and there are some inaccuracies in some of their other articles.

In some cases, such as power failures or system crashes, when the store is determined to be inaccurate, it will be replaced entirely, with a message in the system.log. The "Event Store UUIDs don't match" message sent by Time Machine means it's detected a new Event Store. It's followed by a deep traversal/deep scan, because it can't trust the Event Store.

bernuli wrote:

I don't think it is the directory repair coming from booting in safe mode.

BRumble posted "safe user mode."

My question was, did he/she mean Single User Mode or Safe Mode?

If we can determine that simply booting into Safe Mode is a reliable fix, that's something much safer and more reliable to recommend to the casual user than booting into Single User mode and running UNIX commands.

my guess is BRumble meant "safe mode".

The safe mode trick works when you have months worth of files in the .fseventsd directory.

I agree better to boot in safe mode occasionally, is the best solution. But if you want the best TM performance possible, occasionally blowing away /.fseventsd works well. Just make sure you are backed up first, cause a small typo in single user mode could be the start of a very bad day.

B

bernuli wrote:

. . .

my guess is BRumble meant "safe mode".

That's my guess, too, but I'd like BRumble to clarify.

But if you want the best TM performance possible, occasionally blowing away /.fseventsd works well.

Unless the Event Store is damaged, that really shouldn't make any difference. Time Machine keeps track of the last "event" it processed, so only looks in the Event Store for the directories that have been changed since the previous backup.

If deleting it really improves performance repeatedly, then something else is wrong, and this just deals with the symptom.

You might want to look at the hidden log Time Machine puts in each backup folder -- among other things, it shows the starting Event ID (and the last one is stored in the backup folder as an extended attribute, along with the File System UUID). A clue may be lurking there.

I agree. I think something is wrong with fseventsd. Deleting the its directory deals with the symptom, but I continue to look for a fix.

I'll look at .Backup.log again but I hate doing that, cause they always say "Some filesystem changes made during the course of the backup may not be accounted for. Still busy after 2 retries." and I hate seeing that.

B

Pondini wrote:

bernuli wrote:

. . .

my guess is BRumble meant "safe mode".

That's my guess, too, but I'd like BRumble to clarify.

But if you want the best TM performance possible, occasionally blowing away /.fseventsd works well.

Unless the Event Store is damaged, that really shouldn't make any difference. Time Machine keeps track of the last "event" it processed, so only looks in the Event Store for the directories that have been changed since the previous backup.

If deleting it really improves performance repeatedly, then something else is wrong, and this just deals with the symptom.

You might want to look at the hidden log Time Machine puts in each backup folder -- among other things, it shows the starting Event ID (and the last one is stored in the backup folder as an extended attribute, along with the File System UUID). A clue may be lurking there.

bernuli wrote:

. . .

I'll look at .Backup.log again but I hate doing that, cause they always say "Some filesystem changes made during the course of the backup may not be accounted for.

Aha! Clue! 😉

From the ArsTechnical article above:

As with all kernel-based file system notification mechanisms, including

/dev/fsevents

The FSEvents API includes callbacks for these situations, effectively telling the client, "Unknown changes have occurred. You'll have to do a full rescan yourself, then pick up on the new event stream going forward." That's certainly not what a program wants to hear, but it's the unavoidable truth. and FSEvents is upfront about it. In effect, it's a form of reliability. FSEvents will not lie to you.

Does that ring a bell?

Or, since it says "during the course of the backup," something similar?

Does Time Machine then do a deep traversal?

Message was edited by: Pondini

Huh, that does sort of ring a beep.

One question though, after a standard reboot, should there be files in /.fseventsd that date a month or more back? I guess if it was trying to keep track of removable media than the answer could be yes.

Thanks for the input

B

Pondini wrote:

bernuli wrote:

. . .

I'll look at .Backup.log again but I hate doing that, cause they always say "Some filesystem changes made during the course of the backup may not be accounted for.

Aha! Clue! 😉

From the ArsTechnical article above:

As with all kernel-based file system notification mechanisms, including

/dev/fsevents

, there's still the possibility of file system changes occurring without going through the kernel. For example, a removable disk may be mounted on another non-Leopard computer and modified there. When it returns, the local kernel has no idea what's changed.

The FSEvents API includes callbacks for these situations, effectively telling the client, "Unknown changes have occurred. You'll have to do a full rescan yourself, then pick up on the new event stream going forward." That's certainly not what a program wants to hear, but it's the unavoidable truth. and FSEvents is upfront about it. In effect, it's a form of reliability. FSEvents will not lie to you.

Does that ring a bell?

bernuli wrote:

Huh, that does sort of ring a beep.

One question though, after a standard reboot, should there be files in /.fseventsd that date a month or more back? I guess if it was trying to keep track of removable media than the answer could be yes.

I don't know -- I've never been able to find much documentation on how it works, just the API for how an app should use it (the File System Events Programming Guide, if you're interested).

But each volume (drive or partition) has a separate Event Store, so it should contain only the records for that volume.

Perhaps the Safe Boot examines the store, determines when something bad happened, and deletes the earlier entries.

You answered before my second thought:

Since the message says "during the course of the backup," something similar?

Does Time Machine then do a deep traversal?

I don't have the deep traversal problem other have been having. I do of course end up with

Scanning nodes needing deep traversal

Node requires deep traversal: / reason:must scan subdirs|new event db|

after removing .fseventsd

My problem with Time Machine has been that it backs up loads of directores, for no reason. For instansce, it travels through my iPhoto Library and backs up 50 or so directores, but no files. Even though I never touched iPhoto or had an iphone plugged in etc.

B

You answered before my second thought:

Since the message says "during the course of the backup," something similar?

Does Time Machine then do a deep traversal?

bernuli wrote:

I don't have the deep traversal problem other have been having.

Sorry, what I meant was, when TM sends the "Some filesystem changes made during the course of the backup may not be accounted for" message, does it then do a deep traversal? Seems like it ought to, either during that same backup, or the next one..

My problem with Time Machine has been that it backs up loads of directores, for no reason. For instansce, it travels through my iPhoto Library and backs up 50 or so directores, but no files. Even though I never touched iPhoto or had an iphone plugged in etc.

Hmm. Sounds like those directories are being reported as having something changed, but nothing actually is. So TM backs-up the directory itself, then looks at each folder/file inside it, trying to find which ones need to be backed-up.

Do you have any apps running that might touch a lot of directories without actually changing anything?

Yeah that is what is happening. This will happen on a fresh boot, with nothing running. All I have in login items is Terminal and /Library/Application Support/BlackBerry/BlackBerry Device Manager.app

Since I just cleaned out .fseventsd, I will have to wait a month for TM to degrade then I can go through what is running to see if there is a conflict. But for your reference, I have the usual things running:

me@mbp:~$ ps ax

COMMAND

/sbin/launchd

/usr/libexec/kextd

/usr/sbin/ntpd -c /private/etc/ntp-restrict.conf -n -g -p /var/run/ntpd.pid -f /var/db/ntp.drift

/usr/sbin/cron

/System/Library/PrivateFrameworks/MobileDevice.framework/Versions/A/Resources/us bmuxd -launchd

/usr/sbin/syslogd

/usr/sbin/securityd -i

/System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework/ Support/mds

/usr/sbin/mDNSResponder -launchd

/System/Library/CoreServices/loginwindow.app/Contents/MacOS/loginwindow console

/usr/sbin/KernelEventAgent

/usr/libexec/hidd

/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CarbonCo re.framework/Versions/A/Support/fseventsd

/sbin/dynamic_pager -F /private/var/vm/swapfile

/usr/libexec/dpd

/usr/sbin/diskarbitrationd

/usr/sbin/DirectoryService

/usr/libexec/configd

/usr/sbin/blued

autofsd

/usr/sbin/notifyd

/usr/sbin/distnoted

/System/Library/CoreServices/coreservicesd

/System/Library/Frameworks/ApplicationServices.framework/Frameworks/CoreGraphics .framework/Resources/WindowServer -daemon

/System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/cvmsServ

/sbin/launchd

/System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/cvmsComp_x86_64 1

/usr/sbin/coreaudiod

/sbin/launchd

/System/Library/CoreServices/Dock.app/Contents/MacOS/Dock

/System/Library/CoreServices/SystemUIServer.app/Contents/MacOS/SystemUIServer

/System/Library/CoreServices/Finder.app/Contents/MacOS/Finder

/usr/sbin/pboard

/System/Library/Frameworks/ApplicationServices.framework/Frameworks/ATS.framewor k/Support/fontd

/usr/libexec/UserEventAgent -l Aqua

/System/Library/CoreServices/ServerScanner

/Library/Application Support/Tablet/PenTabletDriver.app/Contents/MacOS/PenTabletDriver

BBLaunchAgent

RimAlbumArtDaemon

/System/Library/CoreServices/CCacheServer.app/Contents/MacOS/CCacheServer

/Applications/Utilities/Terminal.app/Contents/MacOS/Terminal -psn_0_53261

/Library/Application Support/Tablet/PenTabletDriver.app/Contents/Resources/ConsumerTouchDriver.app/C ontents/MacOS/ConsumerTouchDriver -psn_0_77843

/Library/Application Support/Tablet/PenTabletDriver.app/Contents/Resources/TabletDriver.app/Contents /MacOS/TabletDriver -psn_0_81940

/System/Library/Frameworks/QuickLook.framework/Resources/quicklookd.app/Contents /MacOS/quicklookd

/System/Library/CoreServices/backupd.bundle/Contents/Resources/backupd-helper -attach

login -pf me

-bash

login -pf me

-bash

login -pf me

-bash

ps ax

me@mbp:~$

Hmm. Sounds like those directories are being reported as having something changed, but nothing actually is. So TM backs-up the directory itself, then looks at each folder/file inside it, trying to find which ones need to be backed-up

Do you have any apps running that might touch a lot of directories without actually changing anything?

bernuli wrote:

Yeah that is what is happening. This will happen on a fresh boot, with nothing running.

Then it's probably caused by something that ran before the restart.

Since I just cleaned out .fseventsd, I will have to wait a month for TM to degrade then I can go through what is running to see if there is a conflict.

Doubtful it's a standard Apple process -- or lots of folks would have seen it. Since it's intermittent, most likely it's something that only runs occasionally.

One other long-shot possibility, of course, is a problem with your installation of OSX. Have you tried installing the 10.6.8 "combo" update? And/or, a fresh version of OSX?

A few folks with very slow backups on 10.6.8 solved it with a fresh install of OSX, but only going back to 10.6.7, not 10.6.8. No idea why, but that did work for a few.

Pondini wrote:

bernuli wrote:

Yeah that is what is happening. This will happen on a fresh boot, with nothing running.

Then it's probably caused by something that ran before the restart.

No. I restart, then without running anything, I run a backup. Then I run another backup. Each incremental backup can take up to 20 minutes. Looking at the actual items that are backed up (not the links) there are lots and lots of folders with very few files.

This did seem to start after I upgraded to 10.6.8. I reapplied the combo update but it did not fix it.

Possibly a fresh install of OSX would fix it but I would like to know exactly what is going on. But for now I have been clearing .fsevents which always brings the backup down to under a minute.

B

bernuli wrote:

. . .

No. I restart, then without running anything, I run a backup. Then I run another backup.

What I'm suggesting is, the problem may have happened before the restart; you wouldn't see the effect until you run a backup.

Or, of course, something that happens during a restart, which would implicate either OSX or something in /Library/StartupItems, /Library/LaunchAgents, /Library/LaunchDaemons, or ~/Library/LaunchAgents.

Possibly a fresh install of OSX would fix it

That's certainly worth a try.

Pondini wrote:

Do you mean Single User Mode? That's very different from Safe Mode. As far as I can tell, Safe Mode doesn't do anything special to the Event Store, per this article: Mac OS X: What is Safe Boot, Safe Mode?

If Safe Mode really does help, perhaps it's because of the directory check/repair?

Good question. I actually booted into Single User and then into Safe Mode. I didn't delete /.fseventsd/ in either instance, but it definitely had its contents removed and that worked for a while.

Although I've known about these modes for a long time I never needed to log in with them. Indeed, I wouldn't have done it with this, either, but I've been up against this problem since May of this year, and I'm way past my wits end. I'd been relying on CCC until getting a new external and deciding to take another crack at this. It's crazy how nothing has been a permanent solution, and short of starting with a brand new user with a factory-fresh file system, I've tried everything: install updates, fsck'd a million times, reinstalled OS (light version and full version), new external, on and on. No dice. Even this solution only lasted for a few days. Going back through the logs (deleted as of this morning while running Onyx) shows that deep traversals started within one day, though I didn't notice them at first because it was still going relatively quickly. Back up to 7 minute backups now, with 25,000-ish files on the first past, 1700 or so on the second. Doesn't matter if the computer is idle or active, the file count varies only slightly (likely reflecting what actually has changed on top of these 25k files).

By the by, thank you for taking all the time and effort to maintain your site. Although none of the steps resolved my particular issue (basically identical to this thread), I found it extremely informative and well-structured.