Currently Being ModeratedNov 23, 2011 11:28 AM (in response to Mike O\'Brien2)
The system drive was copied from the old drive by Carbon Copy Cloner. Because the new drive (an SSD) is slightly smaller than the original, CCC could not do a block image copy. Whatever it did do, left the resulting file system in good shape according to Disk Utility and TechTool Pro 5, but with something in there that caused Time Machine to "Process" 370,000 items, and to take three hours to back up a 20Mb incremental backup. I have no idea what Time Machine calls an "item", and Apple being Apple, there's no way to find out.
I had to abandon that attempt, and use Disk Utility to copy the drive. It did do a block copy, since there was enough free space on the source partition for it to get away with doing that. That copy works fine with Time Machine, so the problem's solved.
Currently Being ModeratedJan 9, 2013 6:43 PM (in response to Mike O\'Brien2)
This seems to be a problem with recovered systems. I had a new, much bigger HD installed in a MacBook, did a fresh install of Snow Leopard, and restored everything else from a Time Machine backup. Once I got everything up and running, I tried an incremental backup to the original TM archive. Time Machine scanned over 800,000 items (!), "prepared" over 88,000 items, started backing up at a rate of about 100 kb per minute, and finally just stalled at about 22MB (of 230 MB to be backed up.)
I figured that maybe the Spotlight index from the old drive wasn't quite matching up with the restored files on the new HD, so I forced Spotlight to re-index the whole thing. (In Terminal, "sudo mdutil -E /") Didn't help. I had to remove the old backups folders from the TM volume and start from scratch.
You can muck around in the old backup folders with Finder, so they're still useful as an archive; they just don't work with Time Machine. (Wherever you stash them, you'll probably want to exclude them from TM, seeing as they'll double the size of the new TM archive.)
Currently Being ModeratedJan 9, 2013 7:40 PM (in response to jpdemers)
I finally found a likely cause: massive fragmentation of the volume directory. Rebuilding the volume directory with Disk Warrior has so far solved all problems with extremely slow Time Machine backups.
Currently Being ModeratedJan 10, 2013 1:15 PM (in response to Mike O\'Brien2)
Yikes! There must be 2-3 million items in that directory -- how long did the defrag take?
Currently Being ModeratedJan 10, 2013 3:01 PM (in response to jpdemers)
Not long at all, because it doesn't bother with the original directory. It grots over the disk and builds a new directory, which it lets you preview (by mounting it); if you approve it rewrites the new directory over the old one. Works like a charm, at least for me. It's surprisingly robust software.
Currently Being ModeratedJan 10, 2013 5:11 PM (in response to Mike O\'Brien2)
It can figure out which files are trash, without looking at the directory? It has to be cheating to some extent -- peeking at the allocation and extents files at least -- but still, a brand spanking new directory probably solves 95% of all problems that aren't flat-out bad media. With terabyte drives becoming the default, methinks I need (or will need) a copy ... even if it does cost 3x what I paid for the OS.
Currently Being ModeratedJan 10, 2013 7:49 PM (in response to jpdemers)
It doesn't eliminate any files; it just compacts the directory. This seems to cure all of Time Machine's problems, at least for me.
Currently Being ModeratedJan 17, 2013 1:03 PM (in response to Mike O\'Brien2)
I too am plagued by slow TM backups. My log files always say something like "Backed up 89123 files (5MB)" and it takes at least a half an hour. When I look at the backup up files with BackupLoupe I never see close to that many files, maybe a hundred or so.
Were these the same symptoms you had, and did the log files change after your DiskWarrior fix?
Currently Being ModeratedJan 17, 2013 1:10 PM (in response to Jeff Bailey)
Afterwards, the log files looked normal, but I confessI didn't examine them closely, nor did I examine the backup. Once the backup time shrank to normal, I declared success and quit thinking about it.
Currently Being ModeratedJan 20, 2013 7:52 PM (in response to Mike O\'Brien2)
The DiskWarrior trick seems to have made the difference for me, although it didn't happen immediately. Over a few days I had, in no particular order, and often more times than once:
- Erased all files in /.fseventsd/ and restarted the computer
- Re-built the Spotlight index for the drive
- Repaired permissions
- Found some directories with ridiculous numbers of files in them (/private/var/tmp had over 20K files!) and deleted them.
- Rebuilt the directory with DW
It was only after #4 that #5 made the difference. Oddly, for a few backups I was still getting the high file count messages and hour-long backups:
Jan 18 07:16:38 <hostname> com.apple.backupd: Copied 90379 files (8.8 MB) from volume Macintosh HD.
Jan 18 07:50:40 <hostmane> com.apple.backupd: Copied 88790 files (364 KB) from volume Macintosh HD.
Until suddenly I saw this:
Jan 18 09:10:54 <hostname> com.apple.backupd: Copied 92034 files (41.0 MB) from volume Macintosh HD.
Jan 18 09:11:05 <hostname> com.apple.backupd: Copied 112 files (276 bytes) from volume Macintosh HD.
and since then the file counts have been in the hundreds or low thousands, and the backup is complete within a couple of minutes. Hooray!
Currently Being ModeratedJan 21, 2013 5:58 AM (in response to Jeff Bailey)
For those without a copy of Disk Warrior ($99.00), a cheaper solution might be to do a tune-up with OnyX ($0.00): use the "Clean" function to clear out log files and tmp files, then use the "Maintenance" function to rebuild the Spotlight directory.
I think TM backups might continue to run slowly, until the TM directory also gets rebuilt.
Currently Being ModeratedJan 29, 2013 8:39 PM (in response to Jeff Bailey)
Same problem occurred again for no apparent reason. I followed the same procedure and the problem was solved again.
This is wierd.
Currently Being ModeratedJan 31, 2013 12:16 PM (in response to Jeff Bailey)
Re #4 in my previous post: I just discovered that "~/Library/Calendars/Calendar Sync Changes" had over 89,000 files in it, all .tmp files. Excluding that directory has now sped up my TM backups.
Currently Being ModeratedFeb 1, 2013 8:04 AM (in response to Jeff Bailey)
I think I've finally whipped this problem. Booted into single-user mode, enabled mucking with read-only files, and then ran the magic command:
sudo rm -R /.fseventsd/*
Exited, went back into normal mode, and Time Machine went through one more long, slow save. Since then, "done in 90 seconds" has been the norm.
It seems that when this particular directory gets hosed, Time Machine thinks there are changes that it doesn't know about, and does a deep scan in an attempt to find them. (Look up fseventsd in Google and you'll find several discussions about it.) Evidently, restoring to a new hard drive is one of the things that can mess up this directory.
Worth trying, although your mileage may vary.