Time machine backups no longer include hidden system folder like /usr and /sbin

Question

Level 1

0 points

Time machine backups no longer include hidden system folder like /usr and /sbin

My Time Machine backups Aug - Oct seem complete. When I look at them I see all the expected folders and files. Starting in November key system folders disappeared: /sbin, /usr, /bin, etc. However, the non-hidden one are there: /Applications, /Library, etc. Later in November certain things show up again like /usr/share/man/whatis and /usr/share/man/ri, but nothing else under /usr/share. I've looked for all possible places where folders might be excluded and have found nothing.

I'm using Mavericks on a MacBook Pro with a Time Capsule. All software and firmware are completely up to date. I can't explain the behavior. If I take one example I can see /usr in the Nov 5 backup and it is fully populated. It is gone in the Nov 12 backup. In the Nov 19 backup it is back but only with /usr/share/man/whatis. There are no errors in my logs although my logs don't go back that far, so if something happened the week of Nov 5 I don't know.

Anyone with any guesses?

K-Family

MacBook Pro (13-inch Late 2011), OS X Mavericks (10.9)

Posted on Dec 15, 2013 10:20 PM

Reply

Answer 1

db24401

Level 4

1,564 points

Dec 16, 2013 1:14 AM in response to K-Family

I had a similar problem, also happened in November, and I found

my solution in the following links --

https://discussions.apple.com/thread/5125969

http://pondini.org/TM/A4.html

Reply

Answer 2

K-Family Author

Level 1

0 points

Dec 16, 2013 2:47 PM in response to db24401

I had read about this and I'm willing to give it a try tonight. I'm not sure how confident I am it will resovle the issue though. This resolution is most often quoted when the plist file is corrupted with an ExcludeByPath entry. My plist doesn't include that key. I also don't have identical symptoms. My Application, System and Library folders are still showing up in the backups. It is just the hidden system folders like /usr and /sbin that are missing.

Reply

Answer 3

Linc Davis

Level 10

209,739 points

Dec 16, 2013 4:59 PM in response to K-Family

Items that are invisible in the Finder are also invisible in Time Machine. How do you know that's not the issue?

Reply

Answer 4

K-Family Author

Level 1

0 points

Dec 16, 2013 9:19 PM in response to Linc Davis

To give you a concrete example. I run tmutil compare and one of the reported items is:

+ 17.3M /usr/share/vim

I mount my time capsule and go to /Volumes/Time Machine Backups/Backups.backupdb/MacBook/2013-11-05-02402/Macintosh HD/usr/share

I see the vim directory, and pretty much every directory and file I would expect in /usr/share.

If I change directory to:

/Volumes/Time Machine Backups/Backups.backupbd/MacBook/2013-11-16-001559/Macintosh HD/usr/share

The directory is practically empty with just man and ri.

Reply

Answer 5

Linc Davis

Level 10

209,739 points

Dec 16, 2013 10:43 PM in response to K-Family

Please post the output of these commands:

sudo cat /V*/*/B*/*/L*/.ex*.plist

tmutil isexcluded /u*/s*/v*/*

diskutil list

diskutil cs list

Reply

Answer 6

K-Family Author

Level 1

0 points

Dec 17, 2013 7:31 PM in response to Linc Davis

I tried the reset described at http://pondi.org/TM/A4.html and that didn't work. Here is the output of those commands:

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

<dict>

<key>apiExclusionPaths</key>

<array>

<string>//Users/Shared/adi</string>

<string>/Users/Shared/adi</string>

</array>

<key>sourcePaths</key>

<array>

</array>

<key>standardExclusionPaths</key>

<array>

<string>/.MobileBackups</string>

<string>/MobileBackups.trash</string>

<string>/.MobileBackups.trash</string>

<string>/.Spotlight-V100</string>

<string>/.TemporaryItems</string>

<string>/.Trashes</string>

<string>/.com.apple.backupd.mvlist.plist</string>

<string>/.fseventsd</string>

<string>/.hotfiles.btree</string>

<string>/Backups.backupdb</string>

<string>/Desktop DB</string>

<string>/Desktop DF</string>

<string>/Network/Servers</string>

<string>/Library/Updates</string>

<string>/Previous Systems</string>

<string>/Users/Shared/SC Info</string>

<string>/Users/Guest</string>

<string>/private/var/db/com.apple.backupd.backupVerification</string>

<string>/private/var/db/efw_cache</string>

<string>/private/var/db/Spotlight</string>

<string>/private/var/db/Spotlight-V100</string>

<string>/private/var/db/systemstats</string>

<string>/private/var/lib/postfix/greylist.db</string>

<string>/Volumes</string>

<string>/Network</string>

<string>/automount</string>

<string>/cores</string>

<string>/private/tmp</string>

<string>/private/Network</string>

<string>/private/tftpboot</string>

<string>/private/var/automount</string>

<string>/private/var/folders</string>

<string>/private/var/run</string>

<string>/private/var/tmp</string>

<string>/private/var/vm</string>

<string>/private/var/db/dhcpclient</string>

<string>/private/var/db/fseventsd</string>

<string>/Library/Caches</string>

<string>/Library/Logs</string>

<string>/System/Library/Caches</string>

<string>/System/Library/Extensions/Caches</string>

<string>/private/var/log</string>

<string>/private/var/spool/cups</string>

<string>/private/var/spool/fax</string>

<string>/private/var/spool/uucp</string>

</array>

<key>systemFilesExcluded</key>

<key>userExclusionPaths</key>

<array>

<string>/usr/local/McAfee/AntiMalware/var/tmp/LSOFOutput.txt</string>

</array>

</dict>

</plist>

[Included] /usr

[Included] /sbin

[Included] /private/var/agentx

[Included] /private/var/at

[Included] /private/var/audit

[Included] /private/var/backups

[Included] /private/var/db

[Included] /private/var/empty

[Included] /private/var/folders

[Included] /private/var/jabberd

[Included] /private/var/lib

[Included] /private/var/log

[Included] /private/var/mail

[Included] /private/var/msgs

[Included] /private/var/netboot

[Included] /private/var/networkd

[Included] /private/var/root

[Included] /private/var/rpc

[Included] /private/var/run

[Included] /private/var/rwho

[Included] /private/var/spool

[Included] /private/var/tmp

[Included] /private/var/vm

[Included] /private/var/yp

/dev/disk0

#: TYPE NAME SIZE IDENTIFIER

0: GUID_partition_scheme *750.2 GB disk0

1: EFI EFI 209.7 MB disk0s1

2: Apple_HFS Macintosh HD 749.3 GB disk0s2

3: Apple_Boot Recovery HD 650.0 MB disk0s3

No CoreStorage logical volume groups found

Reply

Answer 7

Linc Davis

Level 10

209,739 points

Dec 17, 2013 7:42 PM in response to K-Family

Uninstall the McAfee product by following the instructions on whichever of the pages linked below is applicable:

How to install or uninstall McAfee Internet Security for Mac

How to manually remove VirusScan for Mac 8.6.x using a removal script

How to uninstall and reinstall McAfee Agent 4.x on Macintosh computers

Note that if you have already tried to uninstall the software, you may have to reinstall it in order to finish the job. If you have a different version of the product, the procedure may be different.

Back up all data before making any changes.

Reply

Answer 8

K-Family Author

Level 1

0 points

Dec 17, 2013 7:48 PM in response to Linc Davis

I appreciate the help. Is there something in the output I posted that led you provide this advice or is it just generice "don't use McAfee". I've seen in other threads that McAfee isn't that popular. It just isn't clear from your post if you saw something that made you confident this is the issue of if this is just something good to try.

Thanks

Reply

Answer 9

Linc Davis

Level 10

209,739 points

Dec 17, 2013 9:09 PM in response to K-Family

McAfee is certainly capable of disrupting Time Machine, just as it's capable of disrupting everything else. Since it's completely worthless and will never protect you from anything, there's no reason not to remove it. As long as it's installed, your system is unmaintainable. If you prefer to keep it, refer whatever problems you have to the developer.

Reply

Answer 10

K-Family Author

Level 1

0 points

Dec 17, 2013 10:26 PM in response to Linc Davis

It was worth a shot. I uninstalled McAfee. I then unselected my backup drive, deleted the Time Machine plist file and rebooted. When it came back up I re-selected by backup drive and waited the 120 seconds for the backup to start. No joy - same issue. It is so bizarre. My next thought would be to completely re-initialize the time capsule but this feels like an issue with the Macbook and not the time capsule, so I'm not confident that will help.

Reply

Answer 11

Linc Davis

Level 10

209,739 points

Dec 18, 2013 8:59 AM in response to K-Family

The tmutil command that you ran yesterday was not the one I asked for. Please try again.

Reply

Answer 12

K-Family Author

Level 1

0 points

Dec 18, 2013 6:14 PM in response to Linc Davis

I must have caught a space wrong or something. I tried a copy/paste:

Last login: Tue Dec 17 22:50:47 on console

Stevens-MacBook-Pro:~ Dad$ tmutil isexcluded /u*/s*/v*/*

[Included] /usr/share/vim/vim73

[Included] /usr/share/vim/vimrc

Reply

Answer 13

Linc Davis

Level 10

209,739 points

Dec 18, 2013 11:43 PM in response to K-Family

If you have more than one user account, these instructions must be carried out as an administrator.

Launch the Console application in any of the following ways:

☞ Enter the first few letters of its name into a Spotlight search. Select it in the results (it should be at the top.)

☞ In the Finder, select Go ▹ Utilities from the menu bar, or press the key combination shift-command-U. The application is in the folder that opens.

☞ Open LaunchPad. Click Utilities, then Console in the icon grid.

Make sure the title of the Console window is All Messages. If it isn't, select All Messages from the SYSTEM LOG QUERIES menu on the left. If you don't see that menu, select

View ▹ Show Log List

from the menu bar.

Enter the word "Starting" (without the quotes) in the String Matching text field. You should now see log messages with the words "Starting * backup," where * represents any of the words "automatic," "manual," or "standard." Note the timestamp of the last such message that corresponds to an abnormal backup. Now

CLEAR THE WORD "Starting" FROM THE TEXT FIELD

so that all messages are showning, and scroll back in the log to the time you noted. Select the messages timestamped from then until the end of the backup, or the end of the log if that's not clear. Copy them to the Clipboard by pressing the key combination command-C. Paste (command-V) into a reply to this message.

If all you see are messages that contain the word "Starting," you didn't clear the text field.

If there are runs of repeated messages, post only one example of each. Don't post many repetitions of the same message.

When posting a log extract, be selective. Don't post more than is requested.

Please do not indiscriminately dump thousands of lines from the log into this discussion.

Some personal information, such as the names of your files, may be included — anonymize before posting.

Reply

Answer 14

K-Family Author

Level 1

0 points

Dec 19, 2013 9:09 PM in response to Linc Davis

I've done a closer look at my backups. Using tmutil compare on the last 5 or so it seems like the problem isn't with all the hidden directories. It is just with /usr/share. In the last 5 backups it is /usr/share that shows as missing. Except /usr/share/man/whatis and /usr/share/ri.

Here are the log entries.

12/19/13 5:35:59.290 PM com.apple.backupd[3517]: Starting automatic backup

12/19/13 5:35:59.000 PM kernel[0]: nspace-handler-set-snapshot-time: 1387503361

12/19/13 5:35:59.323 PM com.apple.mtmd[38]: Set snapshot time: 2013-12-19 17:36:01 -0800 (current time: 2013-12-19 17:35:59 -0800)

12/19/13 5:35:59.642 PM com.apple.backupd[3517]: Attempting to mount network destination URL: afp://<name>;AUTH=SRP@<string>%20AirPort%20Time%20Capsule._afpovertcp._tcp.loca l/Data

12/19/13 5:35:59.853 PM NetAuthSysAgent[3519]: TUAMHandler::SetUAMType setting UAMType to 13

12/19/13 5:36:01.253 PM loginwindow[43]: CoreAnimation: warning, deleted thread with uncommitted CATransaction; set CA_DEBUG_TRANSACTIONS=1 in environment to log backtraces.

12/19/13 5:36:30.995 PM com.apple.backupd[3517]: Mounted network destination at mount point: /Volumes/Data using URL: afp://Steven%20Kilby;AUTH=SRP@Kilby%20AirPort%20Time%20Capsule._afpovertcp._tcp .local/Data

12/19/13 5:36:30.998 PM mds[39]: (Normal) Volume: volume:0x7fc13216d000 ********** Bootstrapped Creating a default store:0 SpotLoc:(null) SpotVerLoc:(null) occlude:0 /Volumes/Data

12/19/13 5:36:42.000 PM kernel[0]: hfs: mounted Time Machine Backups on device disk2s2

12/19/13 5:36:43.364 PM com.apple.backupd[3517]: Disk image /Volumes/Data/<computer>.sparsebundle mounted at: /Volumes/Time Machine Backups

12/19/13 5:36:44.347 PM com.apple.backupd[3517]: Backing up to /dev/disk2s2: /Volumes/Time Machine Backups/Backups.backupdb

12/19/13 5:36:58.200 PM com.apple.backupd[3517]: Will copy (4.8 MB) from Macintosh HD

12/19/13 5:36:58.201 PM com.apple.backupd[3517]: Found 59 files (4.8 MB) needing backup

12/19/13 5:36:58.218 PM com.apple.backupd[3517]: 1.98 GB required (including padding), 2.27 TB available

12/19/13 5:37:33.148 PM com.apple.backupd[3517]: Copied 63 items (4.8 MB) from volume Macintosh HD. Linked 718.

12/19/13 5:37:43.555 PM com.apple.backupd[3517]: Will copy (1 KB) from Macintosh HD

12/19/13 5:37:43.555 PM com.apple.backupd[3517]: Found 11 files (1 KB) needing backup

12/19/13 5:37:43.556 PM com.apple.backupd[3517]: 1.98 GB required (including padding), 2.27 TB available

12/19/13 5:37:44.214 PM com.apple.backupd[3517]: Copied 15 items (1 KB) from volume Macintosh HD. Linked 203.

12/19/13 5:37:51.282 PM com.apple.backupd[3517]: Created new backup: 2013-12-19-173750

12/19/13 5:37:51.555 PM com.apple.backupd[3517]: Starting post-backup thinning

12/19/13 5:37:55.117 PM com.apple.backupd[3517]: Deleted /Volumes/Time Machine Backups/Backups.backupdb/<computer>/2013-12-18-172057 (1.9 MB)

12/19/13 5:37:58.114 PM com.apple.backupd[3517]: Deleted /Volumes/Time Machine Backups/Backups.backupdb/<computer>/2013-12-18-162043 (4.1 MB)

12/19/13 5:37:58.115 PM com.apple.backupd[3517]: Post-backup thinning complete: 2 expired backups removed

12/19/13 5:37:58.252 PM com.apple.backupd[3517]: Backup completed successfully.

12/19/13 5:38:03.000 PM kernel[0]: hfs: unmount initiated on Time Machine Backups on device disk2s2

12/19/13 5:38:04.511 PM com.apple.backupd[3517]: Ejected Time Machine disk image: /Volumes/Data/<computer>.sparsebundle

12/19/13 5:38:04.000 PM kernel[0]: AFP_VFS afpfs_unmount: /Volumes/Data, flags 0, pid 3548

12/19/13 5:38:04.000 PM kernel[0]: ASP_TCP Disconnect: triggering reconnect by bumping reconnTrigger from curr value 0 on so 0xffffff802a9a0ca0

12/19/13 5:38:04.000 PM kernel[0]: ASP_TCP Detach: Reply queue not empty?

12/19/13 5:38:04.859 PM com.apple.backupd[3517]: Ejected Time Machine network volume.

Reply

Answer 15

Linc Davis

Level 10

209,739 points

Dec 19, 2013 10:08 PM in response to K-Family

Triple-click anywhere in the line of text below on this page to select it:

com_apple_backup_excludeItem=com.apple.backupd

Copy the selected text to the Clipboard by pressing the key combination command-C.

In the Finder, press the key combination command-F to open a search window, or select

File ▹ Find

from the menu bar. In the search window, select

Search: This Mac

from the row of tokens below the toolbar. Below that is a popup menu initially showing Kind. From that menu, select Other...

A sheet will drop down. In that sheet, select Raw Query and click OK or press return.

Now there will be a text box to the right of popup menu. Click in that box and paste (command-V).

The search window will show all files and folders on mounted volumes that Time Machine excludes from backup, not including the ones excluded in the preference pane. If there is anything in the window that you think shouldn't be excluded, post for further instructions.

Reply