Time Machine intermittently backs up a large amount unnecessarily

Question

sydeny2000 Author

Level 1

0 points

Time Machine intermittently backs up a large amount unnecessarily

SometimesTime Machine backs up 20 GB to 30 GB+ of data after initially finding a much smaller amount to back up.

Each time this happens, the back up goes something like this:

Time Machine starts and finds X MB to back up
The back up proceeds with status messages like "Backing Up: Y MB of X MB" until Y = 0.9 * X, i.e. 90% of X
Then the back up continues with X increasing and Y always 90% of X until it's backed up a large amount of data, usually well over 20 GB.
The back up finishes successfully and Time Machine cleans up.
I run more than one Time Machine back up volume (more on this below) with Time Machine cycling through them. I can see that the back ups immediately before and immediately after this one, both to the other back up volume, do not back up this large amount of data. This indicates that for those back ups, Time Machine does not think that files have changed.
When I check the large back up just made with TimeTracker, I can see that many, if not most, of the files backed up certainly would not have changed since the prior back up to this volume and, in fact, would not have changed since the last system update. It's backing up, for example, most of the files in /Applications

The multiple back up volumes I have, which Time Machine is configured to cycle through, are:

A Time Capsule permanently on the network
A FW800 external drive permanently attached to the MacBook
A FW800 external "mobile" drive I take on the road for back ups and which I periodically attach when not on the road.

MacBook Pro, OS X Mountain Lion (10.8.2)

Posted on Apr 11, 2013 10:50 PM

Reply

Answer 1

sydeny2000 Author

Level 1

0 points

Apr 11, 2013 11:21 PM in response to sydeny2000

So Apple Discussions went away for an update after I accidentally submitted this and before I updated it ...

Here is the full text again:

SometimesTime Machine backs up 20 GB to 30 GB+ of data after initially finding a much smaller amount to back up.

I'd like to know if anyone's experienced this and has a solution to stop it happening.

Each time this happens, the back up goes something like this:

Time Machine starts and finds X MB to back up
The back up proceeds with status messages like "Backing Up: Y MB of X MB" until Y = 0.9 * X, i.e. 90% of X
Then the back up continues with X increasing and Y always 90% of X until it's backed up a large amount of data, say Z GB, usually well over 20 GB.
The back up finishes successfully and Time Machine cleans up.
I run more than one Time Machine back up volume (more on this below) with Time Machine cycling through them. I can see that the back ups immediately before and immediately after this one, both to the other back up volume, do not back up this large amount of data. This indicates that for those back ups, Time Machine does not think that files have changed.
When I check the large back up just made with TimeTracker, I can see that many, if not most, of the files backed up certainly would not have changed since the prior back up to this volume and, in fact, would not have changed since the last system update. It's backing up, for example, most of the files in /Applications.

For example, I might see Backing Up: 0 MB of 7.8 MB, Backing Up: 1.01 MB of 7.8 MB, ..., Backing Up: 7.02 MB of 7.8 MB, Backing Up: 9 MB of 10 MB, ... Backing Up: 4.20 GB of 4.62 GB, ... etc.The multiple back up volumes I have, which Time Machine is configured to cycle through, are:

A Time Capsule permanently on the network
A FW800 external drive permanently attached to the MacBook
A FW800 external "mobile" drive I take on the road for back ups and which I periodically attach when not on the road.

There is a side-effect, which is quite annoying: if a back up volume has more space on it than X and less than Z, then the back ups will repeatedly fail until I manually delete enough old back ups for it to succeed.

This is the log for a recent incident, for just the session that backed up 23.46 GB after starting at 7.8 MB (and I have a number of other similar logs, as well as logs of the sessions before and after):

11/04/13 11:14:31.545 AM com.apple.backupd[45224]: Starting automatic backup

11/04/13 11:14:36.359 AM com.apple.backupd[45224]: Backup failed with error: 19

11/04/13 11:14:36.360 AM com.apple.backupd[45224]: Starting automatic backup

11/04/13 11:14:37.797 AM com.apple.backupd[45224]: Backing up to: /Volumes/Spare A/Backups.backupdb

11/04/13 11:14:42.420 AM com.apple.backupd[45224]: Using file event preflight for Macintosh HD

11/04/13 11:14:43.898 AM com.apple.backupd[45224]: Will copy (7.8 MB) from Macintosh HD

11/04/13 11:14:43.906 AM com.apple.backupd[45224]: Found 301 files (7.8 MB) needing backup

11/04/13 11:14:43.934 AM com.apple.backupd[45224]: 4.01 GB required (including padding), 347.58 GB available

11/04/13 12:14:41.322 PM com.apple.backupd[45224]: Copied 2.74 GB of 2.74 GB, 53481 of 53481 items

11/04/13 1:14:41.920 PM com.apple.backupd[45224]: Copied 3.98 GB of 3.98 GB, 109326 of 109326 items

11/04/13 2:14:42.668 PM com.apple.backupd[45224]: Copied 4.18 GB of 4.18 GB, 142007 of 142007 items

11/04/13 3:14:43.486 PM com.apple.backupd[45224]: Copied 5.01 GB of 5.01 GB, 188912 of 188912 items

11/04/13 4:14:43.520 PM com.apple.backupd[45224]: Copied 11.59 GB of 11.59 GB, 319518 of 319518 items

11/04/13 5:14:43.983 PM com.apple.backupd[45224]: Copied 15.36 GB of 15.36 GB, 444911 of 444911 items

11/04/13 6:14:45.125 PM com.apple.backupd[45224]: Copied 18.01 GB of 18.01 GB, 516604 of 516604 items

11/04/13 7:14:45.710 PM com.apple.backupd[45224]: Copied 20.07 GB of 20.07 GB, 604320 of 604320 items

11/04/13 8:14:46.835 PM com.apple.backupd[45224]: Copied 21.34 GB of 21.34 GB, 690836 of 690836 items

11/04/13 9:14:46.996 PM com.apple.backupd[45224]: Copied 23.46 GB of 23.46 GB, 807804 of 807804 items

11/04/13 10:11:18.239 PM com.apple.backupd[45224]: Copied 917895 files (35.11 GB) from volume Macintosh HD.

11/04/13 10:11:22.082 PM com.apple.backupd[45224]: Using file event preflight for Macintosh HD

11/04/13 10:11:32.163 PM com.apple.backupd[45224]: Will copy (112.9 MB) from Macintosh HD

11/04/13 10:11:32.225 PM com.apple.backupd[45224]: Found 2066 files (191.4 MB) needing backup

11/04/13 10:11:32.227 PM com.apple.backupd[45224]: 5.1 GB required (including padding), 311.09 GB available

11/04/13 10:14:34.814 PM com.apple.backupd[45224]: Copied 21170 files (111.1 MB) from volume Macintosh HD.

11/04/13 10:14:36.686 PM com.apple.backupd[45224]: Created new backup: 2013-04-11-221435

11/04/13 10:15:16.699 PM com.apple.backupd[45224]: Starting post-backup thinning

11/04/13 10:15:37.653 PM com.apple.backupd[45224]: Deleted /Volumes/Spare A/Backups.backupdb/TheMac/2013-04-10-103713 (20.4 MB)

11/04/13 10:15:37.653 PM com.apple.backupd[45224]: Post-back up thinning complete: 1 expired backups removed

11/04/13 10:15:39.327 PM com.apple.backupd[45224]: Backup completed successfully.

Any ideas welcome.

Reply

Answer 2

Linc Davis

Level 10

209,649 points

Apr 12, 2013 9:29 AM in response to sydeny2000

Do you use virtualization software, such as VMware, Parallels, or VirtualBox?

The virtualization software creates a large virtual-disk container that is constantly changing, so it has to be backed up every time Time Machine runs. That will quickly fill up any backup destination.

You should exclude the virtual-disk file(s) from your Time Machine backups. To do that, click the Options button in the Time Machine preference pane. Back up the files on the virtual disk from within the guest system, using a native backup application.

A compromise solution is to create a "snapshot" of the virtual machine in the virtualization software (not a Time Machine snapshot.) That will give you a single large file that never changes and only has to be backed up once. All the subsequent changes will be stored in a new file that's initially much smaller, but will grow over time. You should still exclude that file from TM backup. If you ever need to restore the VM from Time Machine, you'll have a working setup, which will make it easy for you to restore the rest of the data from within the guest system.

Another common cause of large TM snapshots is native OS X disk images. If you store a lot of data on one or more writable disk images, they should be in "sparsebundle" format, which will be backed up much more efficiently than any other format. Also be aware that disk image files are not backed up at all while the image is mounted. Make sure you unmount the image often so it can be backed up. If you ever restore, you will, of course, lose all changes made since the last backup. Consider eliminating large read/write disk images from your workflow, if possible.

Reply

Answer 3

JoeyR

Level 6

8,305 points

Apr 12, 2013 11:43 AM in response to Linc Davis

I was thinking the virtual machine thing too... however, this part of the log above caught my attention:

Copied 917895 files (35.11 GB) from volume Macintosh HD

I would think the virtual disk image would just show up as one large file. As his backup seems to have backed up over 900,000 files, I don't think it would be a VM issue.

Reply

Answer 4

sydeny2000 Author

Level 1

0 points

Apr 13, 2013 12:14 AM in response to JoeyR

Yes, JoeyR, you're right, it isn't a VM issue,

I have a Parallels VM on an external disk and the whole disk is excluded from Time Machine backups, just in case.

Reply

Answer 5

Linc Davis

Level 10

209,649 points

Apr 13, 2013 5:53 AM in response to sydeny2000

The problem is caused by an obscure bug in OS X. I don't know what triggers it.

The only solution that seems to work consistently is to erase the source volume in Disk Utility and restore from a backup.

You should have at least two complete, independent backups of any volume you're going to erase. One backup is not enough to be safe. To erase the boot volume, you need to boot into Recovery (command-R at startup.)

About OS X Recovery

Reply

Answer 6

sydeny2000 Author

Level 1

0 points

Apr 14, 2013 4:35 PM in response to Linc Davis

Thanks, Linc.

I'm a little embarrassed to say I have many more than two complete, independent backups, including two rotating archive copies of the Time Capsule 🙂

I assume that the backup you suggest I restore from should be a Time Machine backup. Some of my backups are image backups done with SuperDuper.

Given that an image is supposed to be close to an exact copy, there would be a higher chance that the trigger for the bug is present in the image backup.

If the erase and restore has fixed this in the past, and the restore was done from a Time Machine backup, then I guess that whatever triggers it is probably in the files that Time Machine doesn't back up.

If a restore from an image backup has fixed it in the past, then there's probably a much smaller set of files in which the trigger could reside, because there are few files skipped by the image backup.

I'll give this a go at the earliest opportunity and update. I'll probably need to give it a month before I'm sure it's not happening at all.

Incidentally, after more than a year of this happening perhaps once every two to four weeks, it's happened about twice in the last few days and has done it on all three Time Machine backups.

Reply

Answer 7

sydeny2000 Author

Level 1

0 points

Jun 19, 2013 1:49 AM in response to Linc Davis

Linc,

This method appears to have worked.

I made a few backups using Time Machine and SuperDuper, with two of each created with the network disconnected.

Booted into Recovery and restored the drive from one of the Time Machine backups. It's been backing up for two weeks now and hasn't exhibited the previous behaviour.

Only two bits of mild weirdness, but they might have been related to the issue I had, so I'll provide some details:

The first Time Machine backup after restoring was before I'd run any apps such as Mail or Safari and to an external drive. Time Machine of course had to do a deep event scan. It found only 1,666 files totalling 180.5MB needing backup, then took 4 hours after which it said it copied 1,567,406 files totalling 181.3MB. Log extract below (Log 1). I surmise that the content of the files wasn't different, but perhaps attributes or something else might have been.
The second Time Machine backup was to the Time Capsule. Still hadn't run anything of note. Again, Time Machine did a deep event scan, but it took a very suspicious zero milliseconds. This time, despite the backup on the Time Capsule being supposedly almost identical to the backup on the external drive, Time Machine found 1,567,361 files totalling 94.36GB requiring backup. After compacting and removing old backups to free space, Time Machine completed the backup in about 10.5 hours, indicating it did indeed copy a lot of data, and reported backing up 1,567,362 files totalling 89.73GB. Then the post-backup thinning took 3 hours deleting one backup of only 3.2GB and the backup verification failed. A later verification passed. Log extract below (Log 2).

Thank you very much for your help. Would be nice if Apple fixed the root cause if this.

regards - David

Log 1

6/06/13 12:23:19.218 PM com.apple.backupd[456]: Deep event scan at path:/ reason:must scan subdirs|new event db|

6/06/13 12:23:19.746 PM com.apple.backupd[456]: First backup after disk inheritance for / - complete scan required

6/06/13 12:51:37.529 PM com.apple.backupd[456]: Finished scan

6/06/13 12:52:38.133 PM com.apple.backupd[456]: Found 1666 files (180.5 MB) needing backup

6/06/13 12:52:38.803 PM com.apple.backupd[456]: 5.34 GB required (including padding), 111.27 GB available

6/06/13 1:23:19.491 PM com.apple.backupd[456]: Copied 54 MB of 180.5 MB, 46288 of 46288 items

6/06/13 2:23:20.357 PM com.apple.backupd[456]: Copied 54.1 MB of 180.5 MB, 424383 of 424383 items

6/06/13 3:23:20.480 PM com.apple.backupd[456]: Copied 58.3 MB of 180.5 MB, 849710 of 849710 items

6/06/13 4:23:20.835 PM com.apple.backupd[456]: Copied 58.4 MB of 180.5 MB, 1016041 of 1016041 items

6/06/13 5:10:06.108 PM com.apple.backupd[456]: Copied 1567406 files (181.3 MB) from volume Macintosh HD.

Log 2

6/06/13 5:20:16.535 PM com.apple.backupd[456]: Forcing deep traversal on source: "Macintosh HD" (mount: '/' fsUUID: 2CC8389F-D5BE-3784-A0E1-EA174609E8DF eventDBUUID: 3162CE9B-83A2-43CC-A825-5AA96D04A3A6)

6/06/13 5:20:16.915 PM com.apple.backupd[456]: Deep event scan at path:/ reason:must scan subdirs|require scan|

6/06/13 5:20:16.915 PM com.apple.backupd[456]: Finished scan

6/06/13 5:24:32.821 PM com.apple.backupd[456]: Found 1567361 files (94.36 GB) needing backup

6/06/13 5:24:38.696 PM com.apple.backupd[456]: 114.72 GB required (including padding), 54.57 GB available

6/06/13 5:25:30.711 PM com.apple.backupd[456]: Ejected Time Machine disk image: /Volumes/Time Capsule/David’s MacBook.sparsebundle

6/06/13 5:25:30.713 PM com.apple.backupd[456]: Compacting backup disk image to recover free space

6/06/13 5:31:04.899 PM com.apple.backupd[456]: Completed backup disk image compaction

6/06/13 5:31:04.900 PM com.apple.backupd[456]: Starting manual backup

6/06/13 5:31:05.009 PM com.apple.backupd[456]: Network destination already mounted at: /Volumes/Time Capsule

6/06/13 5:43:07.764 PM com.apple.backupd[456]: Disk image /Volumes/Time Capsule/David’s MacBook.sparsebundle mounted at: /Volumes/Time Machine Backups

6/06/13 5:43:07.782 PM com.apple.backupd[456]: Backing up to: /Volumes/Time Machine Backups/Backups.backupdb

6/06/13 5:43:23.106 PM com.apple.backupd[456]: Forcing deep traversal on source: "Macintosh HD" (mount: '/' fsUUID: 2CC8389F-D5BE-3784-A0E1-EA174609E8DF eventDBUUID: 3162CE9B-83A2-43CC-A825-5AA96D04A3A6)

6/06/13 5:43:24.085 PM com.apple.backupd[456]: Deep event scan at path:/ reason:must scan subdirs|require scan|

6/06/13 5:43:24.085 PM com.apple.backupd[456]: Finished scan

6/06/13 5:47:26.297 PM com.apple.backupd[456]: Found 1567362 files (94.36 GB) needing backup

6/06/13 5:47:36.413 PM com.apple.backupd[456]: 114.72 GB required (including padding), 66.97 GB available

6/06/13 5:56:48.285 PM com.apple.backupd[456]: Deleted backup /Volumes/Time Machine Backups/Backups.backupdb/David’s MacBook/2013-01-24-084104 containing 3.51 GB; 70.48 GB now available, 114.72 GB required

6/06/13 6:03:35.666 PM com.apple.backupd[456]: Deleted backup /Volumes/Time Machine Backups/Backups.backupdb/David’s MacBook/2013-01-31-111201 containing 825 MB; 71.31 GB now available, 114.72 GB required

6/06/13 6:40:21.208 PM com.apple.backupd[456]: Deleted backup /Volumes/Time Machine Backups/Backups.backupdb/David’s MacBook/2013-02-07-014507 containing 35.06 GB; 106.37 GB now available, 114.72 GB required

6/06/13 6:50:06.398 PM com.apple.backupd[456]: Deleted backup /Volumes/Time Machine Backups/Backups.backupdb/David’s MacBook/2013-02-14-231957 containing 959.6 MB; 107.33 GB now available, 114.72 GB required

6/06/13 6:56:03.880 PM com.apple.backupd[456]: Deleted backup /Volumes/Time Machine Backups/Backups.backupdb/David’s MacBook/2013-02-21-092232 containing 555.7 MB; 107.89 GB now available, 114.72 GB required

6/06/13 7:17:39.672 PM com.apple.backupd[456]: Deleted backup /Volumes/Time Machine Backups/Backups.backupdb/David’s MacBook/2013-02-28-113941 containing 10.59 GB; 118.48 GB now available, 114.72 GB required

6/06/13 7:17:39.672 PM com.apple.backupd[456]: Deleted 6 backups containing 51.51 GB total; 118.48 GB now available, 114.72 GB required

6/06/13 7:17:40.348 PM com.apple.backupd[456]: Copied Zero KB of 94.36 GB, 0 of 1567362 items

6/06/13 8:17:40.825 PM com.apple.backupd[456]: Copied 9.75 GB of 94.36 GB, 241550 of 1567362 items

6/06/13 9:17:41.730 PM com.apple.backupd[456]: Copied 16.2 GB of 94.36 GB, 481855 of 1567362 items

6/06/13 10:17:41.853 PM com.apple.backupd[456]: Copied 20.23 GB of 94.36 GB, 607105 of 1567362 items

6/06/13 11:17:42.124 PM com.apple.backupd[456]: Copied 23.32 GB of 94.36 GB, 689513 of 1567362 items

7/06/13 12:17:48.162 AM com.apple.backupd[456]: Copied 24.64 GB of 94.36 GB, 766476 of 1567362 items

7/06/13 1:17:48.231 AM com.apple.backupd[456]: Copied 31.94 GB of 94.36 GB, 903339 of 1567362 items

7/06/13 2:17:48.458 AM com.apple.backupd[456]: Copied 35.05 GB of 94.36 GB, 1070656 of 1567362 items

7/06/13 3:17:48.552 AM com.apple.backupd[456]: Copied 43.2 GB of 94.36 GB, 1118028 of 1567362 items

7/06/13 4:17:48.754 AM com.apple.backupd[456]: Copied 51.03 GB of 94.36 GB, 1158272 of 1567362 items

7/06/13 5:17:49.716 AM com.apple.backupd[456]: Copied 75.23 GB of 94.36 GB, 1513275 of 1567362 items

7/06/13 5:55:54.306 AM com.apple.backupd[456]: Copied 1567362 files (89.73 GB) from volume Macintosh HD.

7/06/13 5:55:55.360 AM com.apple.backupd[456]: Using file event preflight for Macintosh HD

7/06/13 5:55:59.595 AM com.apple.backupd[456]: Will copy (4.2 MB) from Macintosh HD

7/06/13 5:55:59.606 AM com.apple.backupd[456]: Found 869 files (55.9 MB) needing backup

7/06/13 5:55:59.608 AM com.apple.backupd[456]: 3.87 GB required (including padding), 14.22 GB available

7/06/13 5:57:00.690 AM com.apple.backupd[456]: Copied 6955 files (47.9 MB) from volume Macintosh HD.

7/06/13 5:57:04.104 AM com.apple.backupd[456]: Created new backup: 2013-06-07-055700

7/06/13 5:57:16.734 AM com.apple.backupd[456]: Starting post-backup thinning

7/06/13 9:06:07.514 AM com.apple.backupd[456]: Deleted /Volumes/Time Machine Backups/Backups.backupdb/David’s MacBook/2013-03-07-090104 (3.2 GB)

7/06/13 9:06:07.514 AM com.apple.backupd[456]: Post-back up thinning complete: 1 expired backups removed

7/06/13 9:06:11.532 AM com.apple.backupd[456]: Backup completed successfully.

7/06/13 9:06:25.174 AM com.apple.backupd[456]: Ejected Time Machine disk image: /Volumes/Time Capsule/David’s MacBook.sparsebundle

7/06/13 9:06:25.649 AM com.apple.backupd[456]: Ejected Time Machine network volume.

7/06/13 9:06:25.652 AM com.apple.backupd[456]: Running automatic backup verification.

7/06/13 9:06:25.760 AM com.apple.backupd[456]: Attempting to mount network destination URL: afp://Laure;AUTH=SRP@Hislops-Time-Capsule.local/Time%20Capsule

7/06/13 9:06:26.999 AM com.apple.backupd[456]: Mounted network destination at mount point: /Volumes/Time Capsule using URL: afp://Laure;AUTH=SRP@Hislops-Time-Capsule.local/Time%20Capsule

7/06/13 9:06:30.360 AM com.apple.backupd[456]: Verifying backup disk image.

7/06/13 9:28:07.305 AM com.apple.backupd[456]: CopyHFSMeta failed with termination status: 5

7/06/13 9:28:07.312 AM com.apple.backupd[456]: Backup verification incomplete!

Reply

Answer 8

Linc Davis

Level 10

209,649 points

Jun 19, 2013 6:27 AM in response to sydeny2000

Thanks for the update. Personally, I wouldn't trust a Time Capsule with my data. If it failed a verification, I'd toss it in the recycling bin.

Reply

Answer 9

Pondini

Level 8

38,863 points

Jun 19, 2013 6:36 PM in response to sydeny2000

Is it most of the files in Applications (and/or the top-level System or Library) folders, or the entire folder(s)?

If one or ore of those entire folders, this may be the bug you've been bitten by, and a fix (a temporary one, anyhow): See #D10 in Time Machine - Troubleshooting. Please double-check your backups, and let us know.

Reply

Answer 10

sydeny2000 Author

Level 1

0 points

Jun 19, 2013 7:33 PM in response to Linc Davis

I don't trust any disk with my data. That's why I have so many backups 😁

I would normally restore from an image backup, but given what you said about the likely origin of the problem, decided to restore from the Time Machine backup.

That said, Time Machine backups to Time Capsule have proved very worthwhile in helping out some relatives, young and old. Some younger folk seem to not be able to plug in a FireWire cable 10cm from the laptop port, so Time Capsule being ever-present on the network helps a lot. One older relative managed to delete Safari and empty the Trash. I was able to remote desktop in and restore it from the Time Capsule.

But, as disks get larger, I'm a bit concerned about the unrecoverable bit error rate and the increasing chance that having to read an entire disk to restore means a not-insignificant chance that there's at least one sector you won't be able to read at all. So, if your data is really important, consumer-grade disks might not be good enough.

Reply

Answer 11

sydeny2000 Author

Level 1

0 points

Jun 19, 2013 7:49 PM in response to Pondini

Yes, Pondini, it was most of the files in /Applications, /System and /Library. Sounds like I was bitten by that problem explained in #D10.

However, I've checked the Time Machine volume used most recently and all top-level folders are there for all backups. That Time Machine volume has backups going back to the period in which I was having the problem, but it's been backing up correctly since I erased and restored the disk a couple of weeks ago. I haven't checked the other two Time Machine volumes I use. They'll cycle around on the next backup and the one after that.

Interestingly, the Time Machine backup from which I restored had been affected. All my Time Machine backups were. But it restored OK. The only issues I've seen since were the two items I noted above in my long post and for which I included the log extracts. I've kept all the Time Machine logs up to that point and noticed no other issues.

If there is anything I can provide to help with solving the #D10 problem, please let me know.

Reply

Answer 12

Pondini

Level 8

38,863 points

Jun 19, 2013 7:54 PM in response to sydeny2000

sydeny2000 wrote:

Yes, Pondini, it was most of the files in /Applications, /System and /Library. Sounds like I was bitten by that problem explained in #D10.

No. That bug omits one or more (usually all 3) entire folders.

If there is anything I can provide to help with solving the #D10 problem, please let me know.

No, won't help - it's something different. Thanks anyhow, though.

Reply

Answer 13

Pondini

Level 8

38,863 points

Jun 20, 2013 5:35 AM in response to sydeny2000

Sorry, I didn't mean to just abandon you here. 😟

I agree somewhat with Linc -- I'd be skeptical of the TC, although other things can cause it to fail verification. How old is it? Many seem to fail at 3-4 years.

But that's not necessarily the cause of the files being skipped. We probably can't tell, now, what happened.

There is one thing you might want to do -- see if it happens again, and if so, if you can "catch it" while it's not backing everything up. The only way to tell will be to look at your most recent backup. If some apps or (or other things) are missing, try to see if there's a pattern, such as the apps that come standard with a Mac, instead of the extras like iLife, iWork, Pro apps, and of course 3rd-party apps. Or some other pattern.

In any event, if you see it, try a "full reset" of Time Machine, per #A4 of Time Machine - Troubleshooting, but instead of deleting the plist file, move it somewhere safe, and send it to the address on my Contact page. Note some of the missing items or pattern, and be sure to include either your sydeny2000 alias here or a link to this thread, so I can connect the two. That file may contain a clue.

Reply

Answer 14

sydeny2000 Author

Level 1

0 points

Jun 21, 2013 3:27 AM in response to Pondini

It's OK, I didn't feel abandoned 🙂

But we sort of struck gold on one of the other backups. I entered Time Machine, scrolled back through the backups and found past backups (prior to the erase and restore) in which various top-level folders are missing.

First, let me explain that I have three Time Machine backup volumes:

One on a Time Capsule, shared with other Macs here
Another on a LaCie d2 Quadra external drive
Another on a LaCie Rugged Mobile external drive.

All three backup volumes exhibited this problem of intermittently backing up a large amount.

I restored from backup #3, the LaCie Rugged Mobile drive.

After the restore, none of the backup volumes have the problem.

The volume on which I reported earlier, that had no evidence of missing top-level folders, is the Time Capsule. I've just checked backup #2 and it has missing folders in at least one problem backup on 26 May: it shows only Root, link to User Guides and Information and Users. The backup prior to and before this one show all folders.

I ran TimeTracker and it shows the backup immediately after the 26 May backup is 26.7GB.

It also shows in the 26 May backup only /Users and .com.apple.backupd.mvlist.plist.

In the backup prior to 26 May it shows /usr, /User, /private, /Library, /Applications and .com.apple.backupd.mvlist.plist (one third party app in /Applications).

In the 26.7GB backup after 26 May it shows /usr, /Users, /System, /sbin, /private, /Library, /Developer, /bin, /Applications and .com.apple.backupd.mvlist.plist. /Applications has in it a mix of Apple and third party apps. It totals 13.1GB on the backup, which is probably not quite everything that was in it at the time of the backup. Just guessing on that, but there is 15GB now and I've deleted a few large apps since restoring.

When TimeTracker finally finishes scanning the backup, I'll grab some screen shots.

It also has odd behaviour to do with the /private folder. Backups since the erase and restore all show in Time Machine the /private folder and all other folders. Backups prior to the erase and restore briefly show the /private folder and then it disappears. Interesting, but I don't know that this is the same as the #D10 problem.

I was inclined to run tmutil calculatedrift and maybe tmutil uniquesize, but I don't know if either will give more info and I've never run the latter.

Is any of this helpful?

Reply

Answer 15

Pondini

Level 8

38,863 points

Jun 21, 2013 5:19 AM in response to sydeny2000

sydeny2000 wrote:

. . .

First, let me explain that I have three Time Machine backup volumes:

One on a Time Capsule, shared with other Macs here

Another on a LaCie d2 Quadra external drive

Another on a LaCie Rugged Mobile external drive.

🙂

I ran TimeTracker and it shows the backup immediately after the 26 May backup is 26.7GB.

It also shows in the 26 May backup only /Users and .com.apple.backupd.mvlist.plist.

That does sound like the #D10 bug. But be careful about using Time Tracker to show what was backed-up. It rarely shows /System, for example, as changes to it should be rare; most of the others should have changes, of course.

So look at the actual backups for the combination of what (TM thinks) changed and what was still on your Mac but didn't change.

Backups prior to the erase and restore briefly show the /private folder and then it disappears. Interesting, but I don't know that this is the same as the #D10 problem.

It sure sounds related, if not the same.

I was inclined to run tmutil calculatedrift and maybe tmutil uniquesize, but I don't know if either will give more info and I've never run the latter.

I don't use calculatedrift (Time Tracker or Backup Loupe seem friendlier), and uniquesize will only tell you how large things unique to the selected backup are. Don't think it will help with this.

Now, my question is, what is it doing now? Does your most recent backup have the correct contents?

If not, if you want to send a copy of the /Library/Preferences/com.apple.TimeMachine.plist file to the address on my Contact page, I'd be glad to see if I can spot something. Be sure to identify yourself as sydney2000 and/or include a link to this thread so I know it's you.

And let me know what's missing -- I don't need a big list, just an idea. If, for example, Applications is getting backed-up, but only partially, see if it includes only the standard Apple apps that come with OSX, not extras like iLIfe, iWork, Pro apps, etc., and 3rd-party apps.

Reply