Skip navigation

Slow Time Machine, scans 300,000 items, takes 2 hrs to write 6Mb

961 Views 17 Replies Latest reply: Feb 1, 2013 2:33 PM by jpdemers RSS
1 2 Previous Next
Mike O\'Brien2 Calculating status...
Currently Being Moderated
Nov 21, 2011 2:48 PM

I replaced the system drive in my Mac Pro.  Now Time Machine runs very, very slowly, even after backing up the new drive.  Why is Time Machine first scanning 300,000 items, then taking two hours to back up 6.3 Mb?

Mac Pro, Mac OS X (10.6.8)
  • jpdemers Level 1 Level 1 (20 points)

    This seems to be a problem with recovered systems.  I had a new, much bigger HD installed in a MacBook, did a fresh install of Snow Leopard, and restored everything else from a Time Machine backup.  Once I got everything up and running, I tried an incremental backup to the original TM archive.  Time Machine scanned over 800,000 items (!), "prepared" over 88,000 items, started backing up at a rate of about 100 kb per minute, and finally just stalled at about 22MB (of 230 MB to be backed up.) 

     

    I figured that maybe the Spotlight index from the old drive wasn't quite matching up with the restored files on the new HD, so I forced Spotlight to re-index the whole thing. (In Terminal,  "sudo mdutil -E /")   Didn't help.  I had to remove the old backups folders from the TM volume and start from scratch.  

     

    You can muck around in the old backup folders with Finder, so they're still useful as an archive; they just don't work with Time Machine.  (Wherever you stash them, you'll probably want to exclude them from TM, seeing as they'll double the size of the new TM archive.)

  • jpdemers Level 1 Level 1 (20 points)

    Yikes!  There must be 2-3 million items in that directory -- how long did the defrag take?

  • jpdemers Level 1 Level 1 (20 points)

    It can figure out which files are trash, without looking at the directory?  It has to be cheating to some extent -- peeking at the allocation and extents files at least -- but still, a brand spanking new directory probably solves 95% of all problems that aren't flat-out bad media.  With terabyte drives becoming the default, methinks I need (or will need) a copy ... even if it does cost 3x what I paid for the OS.

  • Jeff Bailey Calculating status...

    Mike:

     

    I too am plagued by slow TM backups.  My log files always say something like "Backed up 89123 files (5MB)" and it takes at least a half an hour.  When I look at the backup up files with BackupLoupe I never see close to that many files, maybe a hundred or so.

     

    Were these the same symptoms you had, and did the log files change after your DiskWarrior fix?

     

    Thanks, Jeff

  • Michael O\'Brien1 Calculating status...

    Afterwards, the log files looked normal, but I confessI didn't examine them closely, nor did I examine the backup.  Once the backup time shrank to normal, I declared success and quit thinking about it.

  • Jeff Bailey Level 2 Level 2 (210 points)

    The DiskWarrior trick seems to have made the difference for me, although it didn't happen immediately.  Over a few days I had, in no particular order, and often more times than once:

    1. Erased all files in /.fseventsd/ and restarted the computer
    2. Re-built the Spotlight index for the drive
    3. Repaired permissions
    4. Found some directories with ridiculous numbers of files in them (/private/var/tmp had over 20K files!) and deleted them.
    5. Rebuilt the directory with DW

     

    It was only after #4 that #5 made the difference.  Oddly, for a few backups I was still getting the high file count messages and hour-long backups:

     

    Jan 18 07:16:38 <hostname> com.apple.backupd[8290]: Copied 90379 files (8.8 MB) from volume Macintosh HD.

    Jan 18 07:50:40 <hostmane> com.apple.backupd[8290]: Copied 88790 files (364 KB) from volume Macintosh HD.

     

    Until suddenly I saw this:

     

    Jan 18 09:10:54 <hostname> com.apple.backupd[9569]: Copied 92034 files (41.0 MB) from volume Macintosh HD.

    Jan 18 09:11:05 <hostname> com.apple.backupd[9569]: Copied 112 files (276 bytes) from volume Macintosh HD.

     

    and since then the file counts have been in the hundreds or low thousands, and the backup is complete within a couple of minutes.  Hooray!


  • jpdemers Level 1 Level 1 (20 points)

    For those without a copy of Disk Warrior ($99.00), a cheaper solution might be to do a tune-up with OnyX ($0.00):  use the "Clean" function to clear out log files and  tmp files, then use the "Maintenance" function to rebuild the Spotlight directory. 

     

    I think  TM backups might continue to run slowly, until the TM directory also gets rebuilt.

  • Jeff Bailey Level 2 Level 2 (210 points)

    Same problem occurred again for no apparent reason.  I followed the same procedure and the problem was solved again.

     

    This is wierd.

  • Jeff Bailey Level 2 Level 2 (210 points)

    Re #4 in my previous post: I just discovered that "~/Library/Calendars/Calendar Sync Changes" had over 89,000 files in it, all .tmp files.  Excluding that directory has now sped up my TM backups.

  • jpdemers Level 1 Level 1 (20 points)

    I think I've finally whipped this problem.  Booted into single-user mode, enabled mucking with read-only files, and then ran the magic command:

     

    sudo rm -R /.fseventsd/*

     

    Exited, went back into normal mode, and Time Machine went through one more long, slow save.  Since then, "done in 90 seconds" has been the norm.

     

    It seems that when this particular directory gets hosed, Time Machine thinks there are changes that it doesn't know about, and does a deep scan in an attempt to find them.  (Look up fseventsd in Google and you'll find several discussions about it.)  Evidently, restoring to a new hard drive is one of the things that can mess up this  directory.

     

    Worth trying, although your mileage may vary. 

1 2 Previous Next

Actions

More Like This

  • Retrieving data ...

Bookmarked By (1)

Legend

  • This solved my question - 10 points
  • This helped me - 5 points
This site contains user submitted content, comments and opinions and is for informational purposes only. Apple disclaims any and all liability for the acts, omissions and conduct of any third parties in connection with or related to your use of the site. All postings and use of the content on this site are subject to the Apple Support Communities Terms of Use.