Time Machine always requires deep traversal?

As mentioned, the first backup after deleting the index will take quite a while. Don't worry about it. See what happens on the next one.

The first one failed half way through (I think because no index). The next one then did a deep traversal (as expected). But so did the next one...

I'll see what happened to it overnight...

ZX48 wrote:

The first one failed half way through (I think because no index).

Please post the messages, so we have some clue to why it failed. Could be any of a number of things.

The next one then did a deep traversal (as expected).

Did it complete successfully?

But so did the next one...

That would be normal if the preceeding one failed or was cancelled. Once TM decides it needs to do a deep traversal, it will be repeated on each backup attempt until one completes successfully.

I'll update tomorrow when back at the desktop machine....

I would not mess with the .Spotlight-V100 folder unless you FIRST turn off indexing for that drive. Either with the Privacy setting in Spotlight preferences or the command:

sudo mdutil -i off /Volumes/NameOfYourHardDriveVolume.

B

The external was already excluded from Spotlight usign the privacy settings. But as Pondini said, it still has a V100 folder due to the essentila indexing that takes place for TM to work properly.

Tm has now recreated the V100 folder.

Here are the last 4 backups inititiated AFTER dleting the v100 folder. I also include the last backup done BEFORE deleting the V100 folder (at 13:49). As I hope you can see - for unknwon reasons, the first backup cancelled (I am pretty sure I did not cause that manually, but it is possible I suppose). Every subsequent backup now goes through a deep traversal durng both passes. One of the backups probably paused when I left the office arounf 17:30 and finished when the computer wakes in the night. Interestingly, despite the node traversal, the total time is barely taking any longer than before (only about 2 minutes more due to the node traversals).

Is it normal that "no expired backups arre removed because none exist"? Here are the last set of logs lookign for post-backup thinning. TM hasn't removed anything since yesterday morning at 8am. Prior to this it normally removed two expired backups every run.

ZX48 wrote:

. . .

the first backup cancelled

. . .

12/02/2013 16:50:41	mds[61]	(Normal) DiskStore: Creating index for /Volumes/MyBook/Backups.backupdb
12/02/2013 16:51:43	com.apple.backupd[1041]	Error: Flushing index to disk returned an error: 0

That's why it failed -- mds created the index, but TM had a problem writing it to the TM disk. So it's not clear whether there's a valid index there now.

We see that message occasionally -- usually, repairing the disk with Disk Utility will fix it; if not, deleting it will; but since you just did that, and are also having the other problems, it makes me suspect the drive. How old is it? Do you have access to another one you can try?

In the meantime, I deleted again fseventsd in single user mode. Here is what happened after that. The first bacup was delayed while the index was rebuilt. After that, I seem to be back where I was: 6-7 minutes for each backup (30-50k files). More if an expred backup needs deleting.

The backup disk is 3 years old. It is a RAID 1 mirror MyBook (2x1TB). I did recently have problems with it failing to mount. Hence finding myself at this thread in the first place. I fixed by doing a full disk repair. After that backups were taing over an hour (but didn't involve a deep traversal). Then I deleted .fseventsd. And they wnet down to < 20minutes. That's where I am again now.

ZX48,

Looking at the logs, I'm starting to get suspicions of that WD drive. I've had a high percentage of WD my book and Sudio edition (RAID) drives go bad.

For TM backup disks, I think it is better to buy 2 and roate them out occasionally rathern than use a RAID solution. I have several drives that I rotate through, a few LaCie D2 Quadras and some self powered USB G-Technology drives.

Maybe you could even borrow a drive, format it and give it a go. Let us know if a new TM drive helps at all.

B

ZX48 wrote:

. . .

The backup disk is 3 years old. It is a RAID 1 mirror MyBook (2x1TB). I did recently have problems with it failing to mount.

. . .

13/02/2013 09:03:21	com.apple.backupd[345]	Waiting for index to be ready (100)
13/02/2013 09:04:21	com.apple.backupd[345]	Waiting for index to be ready (100)

Sorry, I didn't get the e-mail notice when this was posted. 😟

I agree with bernuli that the health of the drive should be considered.

Three years old isn't ancient, of course, but is getting towards the age when some disks begin to fail. Often, they slow down first -- requiring more retries than usual. Until that gets excessive, there usually aren't any failure messages or anything in the logs to indicate a problem.

That can be caused by mechanical problems (they do have tiny bearings and moving parts, of course), but also the electronics, so even a RAID 1 setup can be slow, especially if either drive is having to do a lot of retries (read-backs comparing to what should have been written).

The mounting problems and many indexing messages add to the uncertainty.

For redundancy, also as bernuli says, two entirely separate drives are usually better than a RAID. Actually, using two different apps is even better. See Time Machine - Frequently Asked Question #27 for more.

I'll keep an eye on the drive. Perhaps it is indeed the reason for the slow backups...

I do have a large enough partition on the macmini HD (not the SSD) that I could try with...

Re: redundancy - I have all important files in DropBox synced between 4 computers. My work computer has the TM backup (for easy access local recovery) and it additionally has a CrashPlanPro sync to our University backup system - which is itself backed up offsite to tape at regular intervals.

The additional layer I could add would be a local TM backup of the home computers (essentially meaning that I would have two independent TM backups of the DropBox data.

Thanks for your input though. For now I am fairly happy with TM backing up just a couple of times a day. It no longer causes the fans to spin up, so it doesn't annoy me any more either.

Hi.

Don't know if its the correct thread so please point me to a better location if I'm wrong.

After upgrading from Snow leopard to Mountain lion some weeks ago Time machine is doing very large backups every time (regulary over 15GB). Before the update the backups were very small (compared to now).

* OSX 10.8.2 (Build 12C60)

* MacBook 13" (late 2009)

* Backup location: network drive on a Synology RackStation

* Backup log (copied from "Time machin buddy"):

Starting automatic backup

Attempting to mount network destination URL: afp://tm@NASimodo._afpovertcp._tcp.local/tm

Mounted network destination at mount point: /Volumes/tm using URL: afp://tm@NASimodo._afpovertcp._tcp.local/tm

Resizing backup disk image from 4.42 TB to 536.77 GB

Could not resize backup disk image - DIHLResizeImage returned 22

Disk image /Volumes/tm/Mampfred.sparsebundle mounted at: /Volumes/Time Machine-Backups

Backing up to: /Volumes/Time Machine-Backups/Backups.backupdb

Using file event preflight for Macintosh HD

Will copy (5.7 MB) from Macintosh HD

Found 604 files (5.7 MB) needing backup

4.65 GB required (including padding), 446.83 GB available

Copied 20.84 GB of 20.84 GB, 374040 of 374040 items

Copied 34.23 GB of 34.23 GB, 770116 of 770116 items

Copied 42.65 GB of 42.65 GB, 1073054 of 1073054 items

Copied 1159041 files (44.87 GB) from volume Macintosh HD.

Using file event preflight for Macintosh HD

Will copy (32.6 MB) from Macintosh HD

Found 690 files (32.6 MB) needing backup

4.9 GB required (including padding), 446.83 GB available

Copied 17826 files (9.99 GB) from volume Macintosh HD.

Created new backup: 2013-02-26-140440

After deleting the files listed by sudo ls -l /.fsevensd in safe mode the next 2 or three backups are of normal size but then the size rapidly increases to 10GB, 20GB or sometimes even >40GB.

Any hints on what I can do to stop Time machine from doing such large backups? Do you need more information from me?

thanks,

Stefan

stBorchert wrote:

. . .

After deleting the files listed by sudo ls -l /.fsevensd in safe mode the next 2 or three backups are of normal size but then the size rapidly increases to 10GB, 20GB or sometimes even >40GB.

See #D4 in Time Machine - Troubleshooting for some common causes. The yellow box there may point to a clue.

Any hints on what I can do to stop Time machine from doing such large backups?

Unless the above leads you to something specific, it's hard to tell. You might want to post the messages from one of the larger backups -- a clue might be lurking there.

This thread is mostly about a few folks on the last version of Snow Leopard, that are having some sort of problem with the File System Event Store. For some (most?) of them, deleting it, as you did, seems to solve the problem.

We have seen a couple of folks having a similar problem on Mountain Lion, with no particular cause or resolution yet. If it reoccurs, as it does for you, it seems like there may be a 3rd-party app that's somehow damaging or corrupting it. Are you running any 3rd-party app that might be monitoring or altering your file system, directories, etc., or doing some sort of "cleaning"? Or any anti-virus apps? (Most likely, that would be something you did not get from the App Store.) If so, try turning it off or deleting it for a while, deleting the Event Store again, and see if the problem continues.

Do you have an external HD you can try as a backup location? If so, see if Time Machine has the same behavior when backing-up to it. That would rule out a problem with your NAS.

Is your Mac covered by AppleCare?

Hi.

Pondini wrote:

See #D4 in Time Machine - Troubleshooting for some common causes. The yellow box there may point to a clue.

Yes, TimeTracker indeed gave me a hint about a large change: its MySQL inside my MAMP. The data file for InnoDB is about 9GB and is changing very often (since I'm developing on my MacBook there are a lot of databases which changes regulary).

It's strange that this hasn't been a problem before I upgraded to Mountain Lion.

On the other side the largest backup made yesterday (42.2GB) includes the complete Application folder. For development purposes I'm running Apache Solr (located in a subfolder of /Applications). Is it possible that this causes the Application folder to marked as changed so it is backed up?

Pondini wrote:

Are you running any 3rd-party app that might be monitoring or altering your file system, directories, etc., or doing some sort of "cleaning"?

The only App that would come to my mind is XtraFinder. I disabled it now.

Pondini wrote:

Is your Mac covered by AppleCare?

No, unfortunately not. I've bought it nearly 4 years ago.

Thanks for your help so far. I will try to see how to prevent MySQL from changing the data file or how to reduce its size and have a look at the next backups.

stBorchert wrote:

. . .

Yes, TimeTracker indeed gave me a hint about a large change: its MySQL inside my MAMP. The data file for InnoDB is about 9GB and is changing very often (since I'm developing on my MacBook there are a lot of databases which changes regulary).

It's strange that this hasn't been a problem before I upgraded to Mountain Lion.

Good question -- just a guess -- did any of that software change when you upgraded to ML?

On the other side the largest backup made yesterday (42.2GB) includes the complete Application folder. For development purposes I'm running Apache Solr (located in a subfolder of /Applications). Is it possible that this causes the Application folder to marked as changed so it is backed up?

Another good question. Again, I'm not familiar with the software, much less how it works, but it shouldn't be doing anything to cause that. You might check with the maker.

The log you posted earlier included the "Using file event preflight..." message, which means TM used the File System Event Store to figure out what changed (see the blue box in How Time Machine works its Magic

for an explanation). Double-check whether everything in the Applications folder was backed-up -- ie, were the Utilities and/or other sub-folders backed-up in full? Those sub-folders wouldn't have been backed-up unless they were also marked as changed.

I will try to see how to prevent MySQL from changing the data file or how to reduce its size

That's likely to be difficult or impossible. 😟 Do you have other backups (always a good idea)? If so, and they're updated fairly often, you could just exclude the file from being backed-up by TM, per Time Machine - Frequently Asked Question #10. If not, consider that, per #27 in the same link.