You can make a difference in the Apple Support Community!

When you sign up with your Apple Account, you can provide valuable feedback to other community members by upvoting helpful replies and User Tips.

Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

Time Machine always requires deep traversal?

Hi all-


For the tl;dr crowd: every time my Mac Pro starts a backup, it does a deep traversal (this has gone on for about two months since I got the drive). I'm backing up to a 2TB USB external drive that is always connected. I've added the backup drive to Spotlight's exclude/privacy list. Yesterday I rebooted the machine using the OSX installation disk and performed disk repairs on both the system drive and the backup drive (neither was found to have any errors). I don't have clamXav or any other AV installed. What else can I try to avoid the deep traversal? Here are the console logs for two back-to-back backups (note that these are just before I got into the office, so I don't think user interaction is forcing the deep traversal):


5/31/12 6:53:32 AM
com.apple.backupd[5425]
Starting standard backup
5/31/12 6:53:33 AM
com.apple.backupd[5425]
Backing up to: /Volumes/Time Machine Backups/Backups.backupdb
5/31/12 6:55:00 AM
com.apple.backupd[5425]
Node requires deep traversal:/ reason:contains changes|must scan subdirs|fsevent|
5/31/12 6:57:11 AM
_spotlight[5455]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 6:57:11 AM
_spotlight[5460]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 6:57:21 AM
com.apple.backupd[5425]
No pre-backup thinning needed: 1.87 GB requested (including padding), 1.36 TB available
5/31/12 6:57:23 AM
_spotlight[5466]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 6:57:23 AM
_spotlight[5471]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 6:57:23 AM
_spotlight[5476]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 6:57:23 AM
_spotlight[5481]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 6:57:34 AM
com.apple.backupd[5425]
Copied 2085 files (17.3 MB) from volume Snow Leopard.
5/31/12 6:59:02 AM
com.apple.backupd[5425]
Node requires deep traversal:/ reason:contains changes|must scan subdirs|fsevent|
5/31/12 7:01:05 AM
_spotlight[5498]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 7:01:05 AM
_spotlight[5503]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 7:01:13 AM
com.apple.backupd[5425]
No pre-backup thinning needed: 1.87 GB requested (including padding), 1.36 TB available
5/31/12 7:01:15 AM
_spotlight[5508]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 7:01:15 AM
_spotlight[5513]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 7:01:15 AM
_spotlight[5518]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 7:01:15 AM
_spotlight[5523]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 7:01:26 AM
com.apple.backupd[5425]
Copied 2085 files (16.9 MB) from volume Snow Leopard.
5/31/12 7:02:57 AM
com.apple.backupd[5425]
Starting post-backup thinning
5/31/12 7:32:41 AM
com.apple.backupd[5425]
Deleted backup /Volumes/Time Machine Backups/Backups.backupdb/warp/2012-05-29-143904: 1.36 TB now available
5/31/12 7:32:41 AM
com.apple.backupd[5425]
Post-back up thinning complete: 1 expired backups removed
5/31/12 7:32:42 AM
com.apple.backupd[5425]
Backup completed successfully.
5/31/12 7:53:33 AM
com.apple.backupd[5706]
Starting standard backup
5/31/12 7:53:33 AM
com.apple.backupd[5706]
Backing up to: /Volumes/Time Machine Backups/Backups.backupdb
5/31/12 7:55:13 AM
com.apple.backupd[5706]
Node requires deep traversal:/ reason:contains changes|must scan subdirs|fsevent|
5/31/12 7:57:26 AM
_spotlight[5735]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 7:57:26 AM
_spotlight[5740]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 7:57:35 AM
com.apple.backupd[5706]
No pre-backup thinning needed: 1.87 GB requested (including padding), 1.36 TB available
5/31/12 7:57:37 AM
_spotlight[5745]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 7:57:37 AM
_spotlight[5750]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 7:57:37 AM
_spotlight[5755]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 7:57:37 AM
_spotlight[5760]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 7:57:48 AM
com.apple.backupd[5706]
Copied 2085 files (18.9 MB) from volume Snow Leopard.
5/31/12 7:59:15 AM
com.apple.backupd[5706]
Node requires deep traversal:/ reason:contains changes|must scan subdirs|fsevent|
5/31/12 8:01:24 AM
_spotlight[5778]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 8:01:24 AM
_spotlight[5783]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 8:01:32 AM
com.apple.backupd[5706]
No pre-backup thinning needed: 1.87 GB requested (including padding), 1.36 TB available
5/31/12 8:01:34 AM
_spotlight[5788]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 8:01:34 AM
_spotlight[5793]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 8:01:34 AM
_spotlight[5798]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 8:01:34 AM
_spotlight[5803]
vol.notice /usr/fl/etc/volume.cfg not found
5/31/12 8:01:43 AM
com.apple.backupd[5706]
Copied 2085 files (18.9 MB) from volume Snow Leopard.
5/31/12 8:03:15 AM
com.apple.backupd[5706]
Starting post-backup thinning
5/31/12 8:31:33 AM
com.apple.backupd[5706]
Deleted backup /Volumes/Time Machine Backups/Backups.backupdb/warp/2012-05-29-123915: 1.36 TB now available
5/31/12 8:31:33 AM
com.apple.backupd[5706]
Post-back up thinning complete: 1 expired backups removed
5/31/12 8:31:35 AM
com.apple.backupd[5706]
Backup completed successfully.


The long version: in March I got a 2TB USB external drive to back up the Mac Pro at my work. I've had good success with Time Machine at home, backing up two MacBooks and a Mac Mini with no problems what-so-ever (it really "just works"), so I was surprised when Time Machine struggled with the Mac Pro. I did a lot of research on the web, and followed as much advice as I could (excluding the backup drive from Spotlight, making sure backups completed uninterrupted, repairing drives, etc.), but I continue to have backups that take almost a full hour (and thus Time Machine is almost constantly running). The actual quantity of data backed up is pretty small (similar to the log above, it's usually 20-50 MB).


During my initial backup I ran into a problem with Git which may be pertitent, but I'm not sure. I was in charge of my company's conversion from Subversion to Git, and in the process I locally cloned a bunch of Git repositories (on the order of 300-400 repos). Since Git uses hard links between local clones, this all fit on my 500GB drive no problem. But Time Machine attempted to back up each directory independently, and the backup ballooned to over 2.5TB and didn't fit on the 2TB external drive. After I excluded the Git repos from the backup, the intial backup completed without any visible problems.


Otherwise the machine is used for pretty standard software development (python, gcc, gdb, git), plus standard office email/web browsing.


I've been digging into this on and off since I first noticed the problem shortly after getting the drive and I'm running out of ideas. Anyone have additional suggestions on how to avoid the deep traversal? Let me know if there's additional useful information I've left out.


Thanks!

Stephen

Mac Pro (Mid 2010), Mac OS X (10.6.8)

Posted on May 31, 2012 6:23 AM

Reply
Question marked as Top-ranking reply

Posted on May 31, 2012 6:03 PM

Stephen Bash wrote:

. . .

5/31/12 6:55:00 AM com.apple.backupd[5425] Node requires deep traversal:/ reason:contains changes|must scan subdirs|fsevent|
...
5/31/12 6:59:02 AM com.apple.backupd[5425] Node requires deep traversal:/ reason:contains changes|must scan subdirs|fsevent|

That's curious. I've rarely seen those reason codes; it looks like there was a very large volume of file system changes, but that usually sends a different message.


Even stranger, it's doing a deep traversal on both passes of the same backup. I don't recall seeing that before, either, so I suspect a problem with the File System Event Store, the hidden log of file changes that OSX keps on each volume. TM normally uses it to see what needs to be backed-up, instead of the deep traversal.


Are you running any apps that routinely make many, many changes to the file system (files or folders added, changed, moved, renamed, etc.)?


5/31/12 7:01:05 AM _spotlight[5503] vol.notice /usr/fl/etc/volume.cfg not found

Spotlight is also very unhappy about whatever that is. I don't have an fl folder in my /usr folder, so have no idea what it is, or whether that's connected to the Time Machine problem, but it sure looks suspicious.


Any idea what it is?

174 replies

Jan 25, 2013 8:21 AM in response to Pondini

Pondini wrote:



All I can suggest is booting into single-user mode and deleting it via:


sudo rm -R /.fseventsd/*


The next backup will be a lengthy one (a deep traversal), but thereafter they should be quick.


That fix works great for me. I don't see anything wrong with doing it from single user mode. If there are any experts on the fseventsd daemon, would love to get an opinion.


An alternative could be disabling and then re-enabling journaling vi Disk Utility. You have to hold the option key down when selecting the file menu, then re-enabling journaling from the file menu. I'll try that and check back. But I think removing .fseventsd is the safest method.



B

Jan 25, 2013 10:41 AM in response to Pondini

Pondini wrote:


bernuli wrote:

. . .

An alternative could be disabling and then re-enabling journaling vi Disk Utility.

I guess it's worth a try, but journalling and the Event Store are two very different things.


Oh, right. I had those 2 things cross referenced. Cycling journaling off then on did nothing to improve TM performance. Thanks for pointing that out.


I did however remove the .fseventsd directory on both my boot volume and TM usb drive. Did this from single user mode. (It is a little tricky to rm on the USB drive from single user mode.)


The big thing is to clear the the fseventsd on the boot drive.


Anyway my backup time went from 7 minutes down to 5 seconds. (11 seconds after starting up Mail, Safari and iTunes).


Good to clear once in a while, otherwise you end up with a large amount of real directories in the backup instead of hard links.


B

Jan 25, 2013 10:51 AM in response to bernuli

bernuli wrote:

. . .

I did however remove the .fseventsd directory on . . . my . . . TM usb drive.

That doesn't seem to be a problem.



The big thing is to clear the the fseventsd on the boot drive.

Yup. 🙂 I still wish there was a reliable way to fix it instead, but that is what OSX does sometimes -- such as after an abnormal shutdown or disconnection:


Jan 23 12:29:22 JPsiMac-10v8.local fseventsd[83]: event logs in /Volumes/S.Test.Backups/.fseventsd out of sync with volume. destroying old logs. (1304040 1 1645432)

Jan 23 12:29:22 JPsiMac-10v8.local fseventsd[83]: log dir: /Volumes/S.Test.Backups/.fseventsd getting new uuid: 2AA0319E-D34B-4200-BADA-2EF73ED0D7EB


Those are on an external HD, of course, but it will do the same on a boot volume if necessary.


Good to clear once in a while, otherwise you end up with a large amount of real directories in the backup instead of hard links.

I don't think it should be done as a regular thing, at least not for most users. It seems to affect only a small number of users, and as far as I can tell, only on Snow Leopard.

Feb 5, 2013 1:11 AM in response to Pondini

OK. I can't seem to get this to work. I logged into single user mode (Command+s). Prompt displayed root


I typed: sudo rm -R /.fseventsd/*


It then asked me if I wanted to delete the file (I assume becasue each was listed as rw------)


I said "y" to each file - it took ages, but eventually I got to the bottom of the list and the prompts stopped.


I reran the command, but the same thing happened. It didn't look like anything was actually deleted... I pressed cntrl+c to stop.


I then restarted as normal and ran terminal list command: It finds 13384 files in .fseventsd


What did I do wrong?


Can you please explain to me precisely what commands I shoudl type in terminal or single user mode?

Is there a way to "accept all"?


Should I have said "Y" to the prompt?

Feb 5, 2013 8:14 AM in response to ZX48

There's an explanation of nearly every command right on your Mac.


Type "man" (MANual page) before the command in Terminal: man rm


As you'll see, the -f means remove without prompting for confirmation.


and since -r means remove all the contents, the /* shouldn't be necessary.


(I copied the earlier command from one of the prior posts; then just now saw the other one.)

Feb 5, 2013 8:30 AM in response to ZX48

OK. That has seemed to clear it out. Same directory now just 8 items in it.


Out of inerest, when i rebooted the fans went crazt for 5-10 seconds.


Just testing with TM now....scanning...1.4 million items...preparing...637 items...backing up 540 of 663.5 MB (VERY FAST!)...now stalled...

Feb 5, 2013 8:30 AM in response to ZX48

Yes, those 2 commands are required. Sorry I should have mentioned that.


When you boot into single user mode, your hard drive comes up as read only, preventing you from modifying or deleting anything. Run those 2 commands then you will be able to clear fseventsd.


rm -rf /.fseventsd/* will then work


let me explain what that does


the rm command is used to delete a single file. by adding -rf to the command will delete a file or directory and everything below it.


Be very careful about what you type after the -rf. If you typed rm -rf / and then hit enter by accident, it will start deleting everything on your hard drive. Use control c to cancel the command at any time.


It may seem confusing, but these steps are totally worth doing IMO.


B

Feb 5, 2013 8:32 AM in response to ZX48

ZX48 wrote:


OK. That has seemed to clear it out. Same directory now just 8 items in it.

Great!


Out of inerest, when i rebooted the fans went crazt for 5-10 seconds.


Just testing with TM now....scanning...1.4 million items...preparing...637 items...backing up 540 of 663.5 MB (VERY FAST!)...now stalled...

Yes, both of those are normal. The first backup will probably be quite slow, even if small.

Feb 6, 2013 12:50 AM in response to Pondini

First backup was not so long actually:


05/02/2013 16:26:48com.apple.backupd[333]Starting standard backup
05/02/2013 16:26:48com.apple.backupd[333]Backing up to: /Volumes/MyBook/Backups.backupdb
05/02/2013 16:26:50com.apple.backupd[333]Event store UUIDs don't match for volume: Snow Leopard SSD
05/02/2013 16:26:50com.apple.backupd[333]Node requires deep traversal:/ reason:must scan subdirs|new event db|
05/02/2013 16:28:44com.apple.backupd[333]No pre-backup thinning needed: 3.90 GB requested (including padding), 46.22 GB available
05/02/2013 16:33:14com.apple.backupd[333]Copied 48694 files (630.4 MB) from volume Snow Leopard SSD.
05/02/2013 16:33:20com.apple.backupd[333]No pre-backup thinning needed: 3.15 GB requested (including padding), 45.59 GB available
05/02/2013 16:33:37com.apple.backupd[333]Copied 1818 files (532 KB) from volume Snow Leopard SSD.
05/02/2013 16:33:45com.apple.backupd[333]Starting post-backup thinning
05/02/2013 16:36:58com.apple.backupd[333]Deleted backup /Volumes/MyBook/Backups.backupdb/SL/2013-01-25-215113: 45.61 GB now available
05/02/2013 16:40:26com.apple.backupd[333]Deleted backup /Volumes/MyBook/Backups.backupdb/SL/2013-01-25-175211: 45.97 GB now available
05/02/2013 16:40:26com.apple.backupd[333]Post-back up thinning complete: 2 expired backups removed
05/02/2013 16:40:27com.apple.backupd[333]Backup completed successfully.


New backups parts are much faster now, but I notice that the system still spends a long time cleaning up old backups (8 minutes). Is this because my backup HD is quite full? (95%).


06/02/2013 08:31:45com.apple.backupd[3271]Starting standard backup
06/02/2013 08:31:45com.apple.backupd[3271]Backing up to: /Volumes/MyBook/Backups.backupdb
06/02/2013 08:31:49com.apple.backupd[3271]No pre-backup thinning needed: 3.15 GB requested (including padding), 47.19 GB available
06/02/2013 08:32:16com.apple.backupd[3271]Copied 1875 files (813 KB) from volume Snow Leopard SSD.
06/02/2013 08:32:17com.apple.backupd[3271]No pre-backup thinning needed: 3.15 GB requested (including padding), 47.19 GB available
06/02/2013 08:32:31com.apple.backupd[3271]Copied 1853 files (567 KB) from volume Snow Leopard SSD.
06/02/2013 08:32:39com.apple.backupd[3271]Starting post-backup thinning
06/02/2013 08:36:30com.apple.backupd[3271]Deleted backup /Volumes/MyBook/Backups.backupdb/SL/2013-01-30-103436: 47.70 GB now available
06/02/2013 08:40:23com.apple.backupd[3271]Deleted backup /Volumes/MyBook/Backups.backupdb/SL/2013-01-30-083936: 48.06 GB now available
06/02/2013 08:40:23com.apple.backupd[3271]Post-back up thinning complete: 2 expired backups removed
06/02/2013 08:40:24com.apple.backupd[3271]Backup completed successfully.


Prior to removing .fseventsd, the total tiem was about 23 minutes or so.

Interestingly, the fseventsd directory already has 32 items in it.


The external also sits there for a long time afterwards churning away. Is that normal?

Feb 6, 2013 1:07 AM in response to ZX48

Wait. I've just realised TM has initiated another backup straight after the one I jsut triggered - that is why the HD is spinning. I'm not sure it is fixed. I haven't done anything in the interim and the number of copied files is now already up to 36766 (21.3 MB). :-(


06/02/2013 08:44:51com.apple.backupd[3374]Starting standard backup
06/02/2013 08:44:51com.apple.backupd[3374]Backing up to: /Volumes/MyBook/Backups.backupdb
06/02/2013 08:45:28com.apple.backupd[3374]No pre-backup thinning needed: 3.18 GB requested (including padding), 48.06 GB available
06/02/2013 08:48:38com.apple.backupd[3374]Copied 36766 files (21.3 MB) from volume Snow Leopard SSD.
06/02/2013 08:48:46com.apple.backupd[3374]No pre-backup thinning needed: 3.15 GB requested (including padding), 48.02 GB available
06/02/2013 08:49:05com.apple.backupd[3374]Copied 2000 files (629 KB) from volume Snow Leopard SSD.
06/02/2013 08:49:15com.apple.backupd[3374]Starting post-backup thinning


Any ideas?


Straight after deleting .fseventsd all I had this in that directory was:


sudo ls -l /.fseventsd


total 8

-rw------- 1 root admin 36 5 Feb 16:23 fseventsd-uuid


Now I have:


sudo ls -l /.fseventsd


total 32

-rw------- 1 root admin 9182 6 Feb 01:42 fc007475e29523e8

-rw------- 1 root admin 36 5 Feb 16:23 fseventsd-uuid


So at 01:42 9182 files (?) were created in this directory. Is that correct?

The only things listed in the console log around 01:42 last night are:


06/02/2013 00:32:19com.apple.backupd[2290]Backup completed successfully.
06/02/2013 00:42:47kernelPM notification cancel (pid 64, InternetSharing)
06/02/2013 00:42:47kernelIOPMrootDomain: idle cancel
06/02/2013 00:52:18kernelPM notification cancel (pid 64, InternetSharing)
06/02/2013 00:52:18kernelIOPMrootDomain: idle cancel
06/02/2013 00:55:49kernelPM notification cancel (pid 64, InternetSharing)
06/02/2013 00:55:49kernelIOPMrootDomain: idle cancel
06/02/2013 00:57:49kernelPM notification cancel (pid 64, InternetSharing)
06/02/2013 00:57:49kernelIOPMrootDomain: idle cancel
06/02/2013 01:02:20kernelPM notification cancel (pid 64, InternetSharing)
06/02/2013 01:02:20kernelIOPMrootDomain: idle cancel
06/02/2013 01:07:50kernelPM notification cancel (pid 64, InternetSharing)
06/02/2013 01:07:50kernelIOPMrootDomain: idle cancel
06/02/2013 01:11:50kernelPM notification cancel (pid 64, InternetSharing)
06/02/2013 01:11:50kernelIOPMrootDomain: idle cancel
06/02/2013 01:12:00kernelCODE SIGNING: cs_invalid_page(0x1000): p=2471[GoogleSoftwareUp] clearing CS_VALID
06/02/2013 01:13:29kernelPM notification cancel (pid 64, InternetSharing)
06/02/2013 01:13:29kernelIOPMrootDomain: idle cancel
06/02/2013 01:21:51kernelPM notification cancel (pid 64, InternetSharing)
06/02/2013 01:21:51kernelIOPMrootDomain: idle cancel
06/02/2013 01:40:54kernelPM notification cancel (pid 64, InternetSharing)
06/02/2013 01:40:54kernelIOPMrootDomain: idle cancel
06/02/2013 02:06:38kernelPM notification cancel (pid 64, InternetSharing)
06/02/2013 02:06:38kernelIOPMrootDomain: idle cancel
06/02/2013 02:08:08kernelPM notification cancel (pid 64, InternetSharing)
06/02/2013 02:08:08kernelIOPMrootDomain: idle cancel
06/02/2013 02:10:44kernelCODE SIGNING: cs_invalid_page(0x1000): p=2690[GoogleSoftwareUp] clearing CS_VALID


Any suggestions would be most appreciated!

Oh, and the backup HD is still churning even though the last backup finished >10 minutes ago

Time Machine always requires deep traversal?

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.