Time Machine preparing WAY too many items

I thought about the security update interaction angle, too. Is there any way to test this theory? I thought at least one of my MBPs was O.K. on OS X 10.6.8 for while...too much time has passed to recall, however. All I can say is that I waited at least a couple months beyond the release of OS X 10.6.8 v1.1 (the final -- final -- release of 10.6.8) before I upgraded all my Macs. And then, I did the upgrades one-by-one over a one month period or so as I am always cautious of OS updates.

I also considered a possible interaction between an Airport firmware update on my TC and OS X 10.6.8. Looks like this is out now that I see this problem is not confined to people using a TC via wifi (my connection method).

I still have Apple Care on one of my MBPs, so I will see if I can get a ticket rolling.

Great thread in any case...I wish I could add more.

I have the problem of Time Machine finding too many files it thinks need to be backed up, running OS 10.9.1. I had the same problem with a prior OS as well, on a different macbook pro, with a different external harddrive.

The console reports that it found > 100,000 files needing to be backed up. Ultimately, it may back up only 200 files.

The only way I have found to address it (now under 10.9.1 or a few years ago under an earlier OS, machine, and external drive) is to place the no_log file in /.fseventsd.

That seems to solve it every time.

If I delete that directory, and restart the machine, the first few backups operate normally. But, over time (a day or sometimes less), the problem eventually returns. And, the return isn't gradual - I'll have a series of backups that happen normally - time machine finds a few hundred files to back up and then actually copies some number close to the value reported in the "Found X files () needing backup" log message.

But, then one backup will suddenly find > 100,000 files needing backup. And, it just grows from there. It never actually backs up that many files, so I can't determine which files it thinks it needs to backup.

I haven't identified anything that I'm doing that is unusual between the normal backups and the first abnormal backup.

I don't know what the effect of not logging in .fseventsd. But, spending > 10 minutes trying to back up a few MB to a directly connected thunderbolt drive because it's spending 10 of those minutes apparently finding files that it doesn't end up copying is unaceptable.

If there a way to get a listing of the files that it thinks need to be backed up (as reported in the "Found X files () needing backup" messages? If I could get that list, I'd have some chance of trying to debug this.

Has anyone found a resolution to this. I followed all the steps and getting over 500,000 being prepared but only a few KBs/MBs actually being backed up.

BTW, this is only happening on one of my three machines running SL.

SomePlaceWarm wrote:

Has anyone found a resolution to this.

Not really, but eventually I have simply "accepted" a few necessary steps as a part of the backup workflow on the affected machine:

1. Backing up to USB disk only, not to Time Capsule.

In the meantime it's just my secondary machine, used only for traveling or to run PPC software. I'm only backing up when I actually use it "on the road". (Other than that, I'm syncing my home folder data using ChronoSync with my new Mac running Mountain Lion, so my files are constantly being backed up to Time Capsule anyway via that Mac.)

2. As soon as the backup starts to take too long, I purge fseventsd before backing up.

For that I'm using a simple AppleScript in the global Script menu:

do shell script "rm -rf /.fseventsd" with administrator privileges

… and then I immediately restart.

Then the "deep traversal" is always much faster than having to wait for tens of thousands "ghost" files being backed up.

But usually the backup time will be acceptable for at least a few days before this procedure becomes necessary.

On a possibly related note, the affected machine also refuses to sleep. Not sure what's causing it and if it's related to this issue, but I've never really investigated it yet. I strongly suspect the poorly programmed Huawei modem driver being the culprit…

Time machine keeps track of files that are changed between backups. If you disconnect your time machine drive- turn time machine off - or have to shut down the mac abnormally (hitting the power button) when you restart - the first backup will recheck all of the files for what changed since the last back up.

Have been running time machine for 2-1/2 years on Snow Leopard - with hourly backups - and had no problem other than when the above listed occurred.

Disconnecting or turning off time machine is not the problem. The problem recurs for many backups when neither of those events has ocurred. And, in the times when it is working nominally, I can disconnect the machine dozens of times without the problem recurring.

We realize that thousands of people have been running time machine for many years without this specific problem occurring. But, several of us are experiencing it and have been for year. It's frustrating that Apple won't take it seriously.

notcloudy = NOT helpful. Read the thread before posting please.

It is so annoying when people report real problems and uses this forum to fix them and someone posts their system is running fine. YIKES.

Lukas,

What is odd is that the long time modality changes if I use rm the fseventsd. TM w.o that command causes the report of 500,000 files prepared but only a small MBs of actual backup. If I rm fseventsd then it reports (TM pref pane and system.log) a small number of KBs but it still takes at least one hour.

BTW, I have reinstalled SL, restored from a TM BU from another (working fine mac) and the problem persists on this one mac. Very odd. I have run disk utility and even the erased the mac and USB disk to zero.

Is there a console log message when TM is doing the deep traverse?

From the apple support site:

Antivirus software can also cause backups to require more time. Make sure your Time Machine backup folder, located on the backup disk and named “Backups.backupdb,” is excluded from virus scanning.

Virus scanning software can make Time Machine backups very slow. If you use Norton AntiVirus or a similar product, try turning it off during backups.

From one whose setup works

I use INTEGO Virus Barrier setup to scan a disk on mount. My Backup drive is only used for Time Machine and is removable - but I just leave it connected and I don't dismount when I shut down. When I start the mac the backup drive mounts before INTEGO starts up - so it does not try to scan the TIme Machine Drive. If I were to dismount the drive to use the connection then plug it back in - INTEGO does start to scan it - but I just stop the scan.

By the way - do you work in a different account than the administrator account.

Wondering if this is an authority problem - in that time machine is looking at all files that may be processed but only backing up accounts you have the authority to change.

Working Time Machine does keep track of files changed somewhere on disk so if you shut down before backup, it backs them up on next backup it may also be an authority issue with who can clear that file.

Have you looked at your Activity Monitory of System Memory. (Go to Utilities Menu - Activity monitor)

If you are showing pages out and swapping it may be that the Time Machine is keeping track of and backing up Virtual Memory processing.

I have 4GB of memory and I do not run any plug ins, services etc. so I don't have any pages going out

As developers get to work on machines with maximum memory - VM and its effect on time machine may have not happened for them.

The point of time machine is to allow you to restore files or an entire system at a particular point in time, deleted files are also tagged and "backed up" - so if you have any processes that create temporary files on hard disk - this would also present as preparing many files and backing up almost nothing.