Time Machine always requires deep traversal?

I've been having the exact same problem with TM for a couple of months now. I've recently earased the external drive I backup to and started TM up again today. The same issue is still continuing. Here's a copy of my log files from this afternoon.

Aug 17 10:46:22 uselgml305562 com.apple.backupd[919]: Starting standard backup

Aug 17 10:46:22 uselgml305562 com.apple.backupd[919]: Backing up to: /Volumes/FailSafe/Backups.backupdb

Aug 17 10:46:22 uselgml305562 com.apple.backupd[919]: Ownership is disabled on the backup destination volume. Enabling.

Aug 17 10:46:26 uselgml305562 com.apple.backupd[919]: Backup content size: 130.5 GB excluded items size: 14.1 GB for volume Macintosh HD

Aug 17 10:46:26 uselgml305562 com.apple.backupd[919]: No pre-backup thinning needed: 139.64 GB requested (including padding), 450.84 GB available

Aug 17 10:46:26 uselgml305562 com.apple.backupd[919]: Waiting for index to be ready (101)

Aug 17 10:46:26 uselgml305562 mds[48]: (Normal) DiskStore: Creating index for /Volumes/FailSafe/Backups.backupdb

Aug 17 11:46:23 uselgml305562 com.apple.backupd[919]: Copied 101.4 GB of 116.4 GB, 280356 of 1128057 items

Aug 17 11:51:41 uselgml305562 com.apple.backupd[919]: Copied 281071 files (112.8 GB) from volume Macintosh HD.

Aug 17 11:51:58 uselgml305562 com.apple.backupd[919]: Node requires deep traversal:/ reason:contains changes|must scan subdirs|fsevent|

Aug 17 11:56:24 uselgml305562 com.apple.backupd[919]: No pre-backup thinning needed: 851.6 MB requested (including padding), 334.70 GB available

Aug 17 11:59:50 uselgml305562 com.apple.backupd[919]: Copied 71475 files (4.8 MB) from volume Macintosh HD.

Aug 17 12:02:10 uselgml305562 com.apple.backupd[919]: Starting post-backup thinning

Aug 17 12:02:10 uselgml305562 com.apple.backupd[919]: No post-back up thinning needed: no expired backups exist

Aug 17 12:02:10 uselgml305562 com.apple.backupd[919]: Backup completed successfully.

Aug 17 12:46:48 uselgml305562 com.apple.backupd[1543]: Starting standard backup

Aug 17 12:46:48 uselgml305562 com.apple.backupd[1543]: Backing up to: /Volumes/FailSafe/Backups.backupdb

Aug 17 12:47:06 uselgml305562 com.apple.backupd[1543]: Node requires deep traversal:/ reason:contains changes|must scan subdirs|fsevent|

Aug 17 12:51:17 uselgml305562 com.apple.backupd[1543]: No pre-backup thinning needed: 834.8 MB requested (including padding), 334.52 GB available

Aug 17 12:55:34 uselgml305562 com.apple.backupd[1543]: Copied 73040 files (16.5 MB) from volume Macintosh HD.

Aug 17 12:55:51 uselgml305562 com.apple.backupd[1543]: Node requires deep traversal:/ reason:contains changes|must scan subdirs|fsevent|

Aug 17 13:00:19 uselgml305562 com.apple.backupd[1543]: No pre-backup thinning needed: 828.6 MB requested (including padding), 334.28 GB available

Aug 17 13:04:38 uselgml305562 com.apple.backupd[1543]: Copied 71586 files (10.2 MB) from volume Macintosh HD.

Aug 17 13:07:28 uselgml305562 com.apple.backupd[1543]: Starting post-backup thinning

Aug 17 13:07:28 uselgml305562 com.apple.backupd[1543]: No post-back up thinning needed: no expired backups exist

Aug 17 13:07:29 uselgml305562 com.apple.backupd[1543]: Backup completed successfully.

Aug 17 13:46:49 uselgml305562 com.apple.backupd[1783]: Starting standard backup

Aug 17 13:46:49 uselgml305562 com.apple.backupd[1783]: Backing up to: /Volumes/FailSafe/Backups.backupdb

Aug 17 13:47:07 uselgml305562 com.apple.backupd[1783]: Node requires deep traversal:/ reason:contains changes|must scan subdirs|fsevent|

Aug 17 13:51:20 uselgml305562 com.apple.backupd[1783]: No pre-backup thinning needed: 822.1 MB requested (including padding), 334.10 GB available

Aug 17 13:56:25 uselgml305562 com.apple.backupd[1783]: Copied 73116 files (4.9 MB) from volume Macintosh HD.

Aug 17 13:56:39 uselgml305562 com.apple.backupd[1783]: Node requires deep traversal:/ reason:contains changes|must scan subdirs|fsevent|

Aug 17 14:01:00 uselgml305562 com.apple.backupd[1783]: No pre-backup thinning needed: 823.5 MB requested (including padding), 334.05 GB available

Aug 17 14:06:43 uselgml305562 com.apple.backupd[1783]: Copied 71559 files (1.6 MB) from volume Macintosh HD.

Aug 17 14:09:36 uselgml305562 com.apple.backupd[1783]: Starting post-backup thinning

Aug 17 14:09:36 uselgml305562 com.apple.backupd[1783]: No post-back up thinning needed: no expired backups exist

Aug 17 14:09:36 uselgml305562 com.apple.backupd[1783]: Backup completed successfully.

Aug 17 14:46:48 uselgml305562 com.apple.backupd[2039]: Starting standard backup

Aug 17 14:46:49 uselgml305562 com.apple.backupd[2039]: Backing up to: /Volumes/FailSafe/Backups.backupdb

Aug 17 14:47:06 uselgml305562 com.apple.backupd[2039]: Node requires deep traversal:/ reason:contains changes|must scan subdirs|fsevent|

Aug 17 14:51:21 uselgml305562 com.apple.backupd[2039]: No pre-backup thinning needed: 841.3 MB requested (including padding), 334.05 GB available

Aug 17 14:56:19 uselgml305562 com.apple.backupd[2039]: Copied 73120 files (16.6 MB) from volume Macintosh HD.

Aug 17 14:56:33 uselgml305562 com.apple.backupd[2039]: Node requires deep traversal:/ reason:contains changes|must scan subdirs|fsevent|

Aug 17 15:00:51 uselgml305562 com.apple.backupd[2039]: No pre-backup thinning needed: 826.0 MB requested (including padding), 334.00 GB available

Aug 17 15:06:32 uselgml305562 com.apple.backupd[2039]: Copied 71564 files (2.8 MB) from volume Macintosh HD.

Aug 17 15:09:28 uselgml305562 com.apple.backupd[2039]: Starting post-backup thinning

Aug 17 15:09:28 uselgml305562 com.apple.backupd[2039]: No post-back up thinning needed: no expired backups exist

Aug 17 15:09:28 uselgml305562 com.apple.backupd[2039]: Backup completed successfully.

Aug 17 15:46:48 uselgml305562 com.apple.backupd[2279]: Starting standard backup

Aug 17 15:46:49 uselgml305562 com.apple.backupd[2279]: Backing up to: /Volumes/FailSafe/Backups.backupdb

Aug 17 15:47:07 uselgml305562 com.apple.backupd[2279]: Node requires deep traversal:/ reason:contains changes|must scan subdirs|fsevent|

Aug 17 15:51:25 uselgml305562 com.apple.backupd[2279]: No pre-backup thinning needed: 824.8 MB requested (including padding), 334.00 GB available

Aug 17 15:56:45 uselgml305562 com.apple.backupd[2279]: Copied 73121 files (2.2 MB) from volume Macintosh HD.

Aug 17 15:57:00 uselgml305562 com.apple.backupd[2279]: Node requires deep traversal:/ reason:contains changes|must scan subdirs|fsevent|

Aug 17 16:01:08 uselgml305562 com.apple.backupd[2279]: No pre-backup thinning needed: 825.0 MB requested (including padding), 333.95 GB available

Aug 17 16:07:14 uselgml305562 com.apple.backupd[2279]: Copied 71564 files (493 KB) from volume Macintosh HD.

Aug 17 16:10:11 uselgml305562 com.apple.backupd[2279]: Starting post-backup thinning

Aug 17 16:10:11 uselgml305562 com.apple.backupd[2279]: No post-back up thinning needed: no expired backups exist

Aug 17 16:10:11 uselgml305562 com.apple.backupd[2279]: Backup completed successfully.

Aug 17 16:46:48 uselgml305562 com.apple.backupd[2585]: Starting standard backup

Aug 17 16:46:49 uselgml305562 com.apple.backupd[2585]: Backing up to: /Volumes/FailSafe/Backups.backupdb

Aug 17 16:47:06 uselgml305562 com.apple.backupd[2585]: Node requires deep traversal:/ reason:contains changes|must scan subdirs|fsevent|

Aug 17 16:51:19 uselgml305562 com.apple.backupd[2585]: No pre-backup thinning needed: 838.9 MB requested (including padding), 333.77 GB available

Aug 17 16:56:27 uselgml305562 com.apple.backupd[2585]: Copied 73126 files (1.7 MB) from volume Macintosh HD.

Aug 17 16:56:41 uselgml305562 com.apple.backupd[2585]: Node requires deep traversal:/ reason:contains changes|must scan subdirs|fsevent|

Aug 17 17:01:00 uselgml305562 com.apple.backupd[2585]: No pre-backup thinning needed: 846.1 MB requested (including padding), 333.73 GB available

Aug 17 17:06:54 uselgml305562 com.apple.backupd[2585]: Copied 71569 files (482 KB) from volume Macintosh HD.

Aug 17 17:10:00 uselgml305562 com.apple.backupd[2585]: Starting post-backup thinning

Aug 17 17:10:00 uselgml305562 com.apple.backupd[2585]: No post-back up thinning needed: no expired backups exist

Aug 17 17:10:01 uselgml305562 com.apple.backupd[2585]: Backup completed successfully.

I see you're running 10.6.8. Did your Mac come with Snow Leopard or earlier, or did it come with Lion?

Most of this thread is about Time Machine doing two deep traversals per backup, on a Mac that came with Lion and was hacked to run Snow Leopard. See the post by Stephen Bash on Jul 17 at 12:47 pm EDT.

We don't know if that was the cause of his problem, but I've never seen it posted before, and he's tried everything to fix it, with no success.

It came with Snow Leopard. I installed Lion and then reverted back to Snow Leopard. The system flip-flop occurred 6 months before the TM problems began occurring.

.

"Node requires deep traversal:/ reason:contains changes|must scan subdirs|fsevent|" is what led me to this discussion.

Ah, ok. Then we'll need to start fresh.

There may be something corrupted in your installation of OSX. You might try installing the "combo" update, per Installing the ''combo'' update and/or Reinstalling OSX. If that doesn't help, installing a fresh version of OSX (that won't disturb anything else), per the same article, might help.

10.6.8 on a 2.5 core 2 duo MacBook Pro:

I never have seen the node requires deep traversal problem, but Time Machine had been backing up way too much (better more than less I guess).

Running incremental backups with no apps running for me was taking around 1 minute and 30 seconds. Not a big deal, but there were 498 directories, 625 files and 38 symlinks being backed up. Seemed like it should have been less.

So I reapplied the 10.6.8 combo update as recommended, and the backup performance degraded. Went up to 24 minutes backing up 20,246 directories, 70 files and 65501 symlinks. Again, the machine was basically idle, and every backup was up to 24 minutes with over 20,000 items being backed up.

Disk utility's verify disk and repair permissions did not help. Also did a full time machine reset, and started with a fresh external FW drive as a backup target, still 24 minutes.

What did help was booting in safe mode. While in safe mode, backups were down to 16 seconds.

After booting up normal, backups still down to 18 seconds with a much more reasonable 119 directories and 21 files.

It seems that booting in safe mode seems to clean out the /.fseventsd directory and perhaps some other cache files. Might be worth doing once in a while.

Now with a few applications running, Mail, Excel Safari, Backups are under a minute.

B

Hi all,

Sorry for the radio silence - ended up on business travel and didn't have time to sort anything out.

Unfortunately, my request to "downgrade" to 10.6.7 was nixed by my IT department, so I had to find an alternate route and just wanted to report on my experience.

I had grabbed a copy of Carbon Copy Cloner prior to reinstalling a couple weeks ago and noticed that it has a scheduled incremental backup feature. After discussing with my IT department, we decided I should give it a shot in place of Time Machine. It's been chugging along for a couple weeks now and so far I'm reasonably satisfied. The incremental backup feature of the latest version works as advertised, and the scheduling is done 'in the background', so it's pretty clean. The 'changed' files are stored in an 'archive' and are recoverable via straight-up drag-n-drop. Obviously not as nicely integrated as TM, but it's workable. The other bonus is that the backup is bootable - so in the event of catastrophic HD failure, I can be up and running in as long as it takes to boot from the external drive.

All of that said, from what I can tell - the underlying problem that is causing TM to bomb is not resolved by switching to CCC - mearly masked by different backup mechanics. I'm still getting ~25mb backups / hr when I've done NO work on the computer - most of the changes happening in small files scattered throughout the machine. However, it DOES work - and doesn't take a long time (always <10 minutes, 5-7 of which is sorting out what's changed).

At any rate - thanks to all who provided insight and tried to help resolve the issue. And for those who can't seem to find a solution but still want similar 'incremental' functionality - checkout Carbon Copy Cloner. It's not the same, but it's better than nothing!

Best,

Chris

Try rebooting and logging in from safe mode, then rebooting normal. I have found this to clear out the /.fseventsd directory and will greatly speed up time machine. Though backup time and amount of unnecessary files increases over time.

Would be interesting to see if the safe mode thing works for CCC as well.

B

I had the same problem for months: each and every backup via TimeMachine took the "needs deep traversal" route. Out of desperation, I opened Terminal and looked into the /.fseventsd/ directory:

sudo ls -la /.fseventsd/

and found that there were file system event logs dating back to ... well, about when my trouble started. On a hunch, I removed the whole content (but not the directory itself):

sudo rm -R /.fseventsd/*

Next step, running TimeMachine Backup by hand (I have it in the menu bar), and it completed in well under 10 seconds (and did backup about 3 MB only).

Rebooted the machine, re-ran TimeMachine, now it took longer because it checked the whole source disk (some 1.4 million files or something) - but still finished the whole process under one minute, copying some 80 MB of data (yeah, been running WoW in between tries, so some configs and stats were altered).

Ever since, TM has been nothing but extremely well-behaved. No more deep scans, no thrashing around on the disk, backups just snap by, and accessing their contents works just fine, too.

I'll keep an eye on it, but here's hoping that with the crud of old fs logfiles gone, TM will keep up its speedy behaviour for the time being. And if not, I know where to kick it 😉

Hopefully this helps some of you who also have these fs deep traversal problems.

Just to add some idle speculation, maybe these old log files should be cleared out periodically by some of those maintenance scripts that the system is supposed to run every day/week/month, and something broke in there yet nobody noticed...? Then again, I've not seen these in a while, does Apple still use them?

-Markus

Affected system: 10.6.8 with all current upgrades, running on a Macbook Pro 15" early 2011 (MacBookPro8,2), 8GB, source: internal self-replaced Seagate Momentus XT 750 disk, backing up locally to a 1,5 TB ext. no-name USB3 disk (attached to USB 2)

Message was edited by: me, because I've been a dork who couldn't bother to add his system configuration...

After lots of research and troubleshooting, I too use the same fix periodicly.

However, I boot into single user mode before clearing the .fseventsd directory.

I assume booting in single user mode is the safest way to remove /.fseventsd/*. I am still not sure if this is a safe thing to do. Though so far I have not noticed any side effects.

B

Affected system:

MacBookPro4,1

10.6.8

stock other than upgrade to 4GB Ram

Wow, after months of searching similar threads, this has done it. After reading this and discovering that there were over 41,000 files in /.fseventsd/, I booted into safe user mode. After it did its thing, I logged out and back in as my normal user account. Presto. /.fseventsd/ is fixed, and backups seem to be back down to the <1 minute mark. I had given up on Time Machine and started using CCC exclusively, but that was not optimal. Happy to have another level of backup protection back!

Glad that worked.

I have found that the safe mode boot only helps when .fseventsd is way out of whack. Overtime my TM backup runs start taking slightly longer. The longer time is the result of Time Machine backing up more and more folders (not files just folders) that do not need a backup. A backup with no files changed might end up running for 5-10 minutes.

In these cases I boot into single user mode, then completly remove /.fseventsd. I have not been able to get confirmation on the ill effects of deleting the /.fseventsd other than the next TM run after doing so takes a long time doing full scan (expected). I am not 100% sure this is an ok thing to do but I end up doing it every couple of weeks to keep things fast.

B

BRumble wrote:

Wow, after months of searching similar threads, this has done it. After reading this and discovering that there were over 41,000 files in /.fseventsd/, I booted into safe user mode.

Do you mean Single User Mode? That's very different from Safe Mode. As far as I can tell, Safe Mode doesn't do anything special to the Event Store, per this article: Mac OS X: What is Safe Boot, Safe Mode?

If Safe Mode really does help, perhaps it's because of the directory check/repair?

I don't think it is the directory repair coming from booting in safe mode.

It does not happen every time, but if .fseventsd has a huge number of files in it, I have seen safe mode resulting in a clean out. This is after a running disk utility and verify reporting no problems.

Or perhaps safe mode allows the fseventsd process to properly take care of it's directory.

B