Please explain Time Machine differences

Question

Level 1

5 points

Please explain Time Machine differences

In order to understand what I'm about to say here, please look back at my previous post called "First Time Machine backup not the same size as my computers HD".

I just did a size compare between each folder at the root level of my Macintosh HD with each folder at the root level of my Time Machine backup of my computer. Here are the results:

*Macintosh HD (On my computer)*
Applications: 5.03 GB
Desktop Items: 68.8 MB
Developer: 2.44 GB
Games: 7.83 GB
Library: 7.21 GB
System: 4.63 GB
User Guides And Information: 4 KB
Users: 16.44 GB

*Macintosh HD (On my external USB 2.0 drive-backup)*
Applications: 5.03 GB
Desktop Items: 68.8 MB
Developer: 2.45 GB
Games: 7.83 GB
Library: 7.41 GB
System: 4.62 GB
User Guides And Information: 4 KB
Users: 15.83 GB

My computer's hard drive is a 160 GB drive but actual capacity is 148.73 GB. I have used about 125.27 GB of it. And I just finished removing lots of stuff that I didn't need or had already backed up. The current configuration I have is what I want Time Machine to backup. I ran Time Machine 2 nights ago and it supposedly finished successfully. There were no errors or anything. After the backup finished, the size of my Macintosh HD on my external drive was 44.69 GB. Can anyone explain the differences? Can anyone tell me how all those numbers above add up to 44.69 GB? Because, when I add them up, I get a number a lot higher than 44.69. I am still trying to figure this out. I want to MAKE SURE everything is backed up before I go wiping my computers hard drive and installing MacOS 10.6 Snow Leopard. Thanks for any help anyone can give me. I am currently running MacOS 10.5.8 Leopard.

MacBook, Mac OS X (10.5.8), 2.2 GHz Intel Core 2 Duo, 2 GB of Memory

Posted on Oct 23, 2010 10:06 AM

Reply

Answer 1

Kappy

Level 10

361,741 points

Oct 23, 2010 10:54 AM in response to Brian30

By my calculation the backup and hard drive difference is 410 MBs which is probably attributable to miscellaneous system files like caches that TM knows should not be backed up. The TM configuration file has a list of items not to be copied.

See User Tips for Time Machine for help with TM problems. Also you can select Mac Help from the Finder's Help menu and search for "time machine" to locate articles on how to use TM. See also Mac 101- Time Machine.

Reply

Answer 2

jsd2

Level 5

6,245 points

Oct 23, 2010 12:48 PM in response to Brian30

when I add them up, I get a number a lot higher than 44.69.

I got 43.65GB for the internal and 43.24GB for the backup.
Did you notice that the size of Desktop Items is 68.8 MB ?

As Kappy said, the differences in the list of files on the internal HD compared to the list on the TM backup are small and explainable. However, there is a large discrepancy within the internal HD itself between the sum of the sizes of the all the listed folders (43.65 GB) and the "used" HD disk space that you reported as 125.27GB. Where is the "missing" 81.6 GB of disk space on your internal HD??

If there are other user accounts, the data in their protected folders (Music, Pictures, Movies, etc) -- will not be accessible to your own user account, and the size of the /Users folder as seen from your own account will be wrong. But unless there are indeed other users with very large quantities of user data, then something is wrong on your HD and is very likely fixable. In the other thread, Lyssa suggested that you run OmniDiskSweeper to try to find what is taking up your disk space - did you do this?

Reply

Answer 3

Pondini

Level 8

38,863 points

Oct 23, 2010 3:55 PM in response to Brian30

The small differences are in your /Library and /Users folders.

As I posted in one of your other threads on this subject, there are some usually-small things that Time Machine omits, such as system work files, trash, etc.

As posted there, see the *What’s excluded from backups* section in #11 of [Time Machine - Frequently Asked Questions|http://web.me.com/pondini/Time_Machine/FAQ.html] (or use the link in *User Tips* at the top of this forum).

Some of the items omitted are in hidden folders, mostly the /private folder. If you're having certain problems, which you may find if you follow the instructions in the blue box of #11 in the FAQ, they may explain both the extra space used on your internal HD and why your backups are so much smaller than the amount used on your internal HD.

Reply

Answer 4

Brian30 Author

Level 1

5 points

Oct 25, 2010 1:15 AM in response to jsd2

I don't know if you will see this response or not but replying anyways. To answer your questions, I don't know where that 81.6 GB of internal hd space is. There are no other user accounts on my computer. I am the sole user of this computer. If there is something wrong on my internal hard drive, what do you think it would be? I ran Disk Utility and it did find some errors that it repaired. Don't think that is going to change anything though. Yes, I did run OmniDiskSweeper. It shows the folders that I CAN see and their sizes. It also shows folders that I CAN'T see and their sizes. And some of those folders that I can't see are rather large. Like, over 1 GB in size. Why are there so many invisible files? And how many of them are actually safe to delete? If you do read this, I'll await your response. Thanks.

Reply

Answer 5

jsd2

Level 5

6,245 points

Oct 25, 2010 4:36 AM in response to Brian30

I ran Disk Utility and it did find some errors that it repaired.

Just to be sure - are you talking about Disk Utility's "Repair Disk", or "Repair Disk Permissions"? "Repair Disk" is the important one in this situation, but it cannot be run on the disk that you are booted from - did you run it from somewhere else? If you didn't run Repair Disk, then run "Verify Disk" now. You can run this from your startup disk to be sure there are no errors. If it finds any then you will still have to run "Repair Disk" from somewhere else, such as your Install disc.

Yes, I did run OmniDiskSweeper. It shows the folders that I CAN see and their
sizes. It also shows folders that I CAN'T see and their sizes. And some of those
folders that I can't see are rather large. Like, over 1 GB in size. Why are there so
many invisible files? And how many of them are actually safe to delete?

The whole point of OmniDisk sweeper is to organize your folders and files by size, invisible or not, so that you can see how your disk space is being used. You cannot delete anything unless you know exactly what you are deleting and why you don't need it. Deleting something is the last step, not the first step.

First you need to find the "missing" disk space. When you add your visible top-level folders in Finder, you get 43.6GB as a sum, and you had reported that the "used" space when you did a GetInfo on Macintosh HD was 125.27GB, for a discrepancy of 81.6GB. We now want to do the same comparison in OmniDiskSweeper, which can see folders and files that are invisible to Finder. The top-level folders will be at the leftmost column of OmniDiskSweeper's display of Macintosh HD, sorted by size, with the sum of all the sizes given at the top of that window. OmniDiskSweeper's "Drive List" window will give the total capacity and utilization as reported by the HD. Normally the sum of the sizes of items in the main window should be close to the total "used" space in the "Drive List" window. In my own system, I got the following:
.
!http://i51.tinypic.com/2nvv1tv.jpg!

The numbers don't match exactly, because I have other user accounts that OmnIDiskSweeper can't measure from my current account , and there are also some system folders that have restricted permissions,. The numbers are pretty close, though: 136.5GB used, of which 135.1GB is "accounted for". What do you see when you run OmniDiskSweeper in your own system? If there is not a big discrepancy, then look in the left-hand column for the 81GB that was not seen when you used Finder.

Reply

Answer 6

Pondini

Level 8

38,863 points

Oct 25, 2010 7:50 AM in response to Brian30

Brian30 wrote:
. . .
It also shows folders that I CAN'T see and their sizes. And some of those folders that I can't see are rather large. Like, over 1 GB in size. Why are there so many invisible files? And how many of them are actually safe to delete? If you do read this, I'll await your response. Thanks.

I'll try this one last time:

Some of the usually-small items omitted from Time Machine are in hidden folders, mostly the /private folder. If you're having certain problems, those items may get quite large. Most likely, the "missing" data is there, and also explains the difference in the size of your backups. See the blue box in #11 of [Time Machine - Frequently Asked Questions|http://web.me.com/pondini/Time_Machine/FAQ.html] (or use the link in *User Tips* at the top of this forum).

Just the other day, a user found 98 GB that way . . .

Reply

Answer 7

Brian30 Author

Level 1

5 points

Oct 25, 2010 8:58 PM in response to jsd2

I am talking about both Disk Utility's "Repair Disk" AND "Repair Disk Permissions" scans. I ran both scans on my computer. Both scans did do repairs on my computer. I had restarted my computer and started it up using the install disc that came with my computer. I then opened Disk Utility and ran both of those scans. After both scans completed and repairs finished, I restarted my computer using the internal hard drive as the startup disc and ejected the disc.

I ran OmniDiskSweeper again and this is what it came up with. I've been trying to "paste" in an image of my desktop the way you did, but not having any luck. But anyway, here's the results:

*Drive List window*
Macintosh HD 23.5 GB free, 125.0 GB used, 148.7 GB total (if the numbers seem different now, its because I copied my "Desktop Items" folder to my external drive and deleted it off my computer.

*Macintosh HD window*
124.8

I hope these answer your questions.

Reply

Answer 8

Brian30 Author

Level 1

5 points

Oct 25, 2010 10:41 PM in response to Pondini

I opened the /private folder by going under the "Go" menu and selecting "Go to Folder" option. There is 4 folders inside it:

etc - 6.6 MB
tftpboot - 0 KB
tmp - 0 KB
var - 79.2 GB

Just by simply adding that all up, it comes to 85.8. Now, I am not sure if that is the correct way to add that or not. But either way, that is a bigger number than 81.6 GB. I added up the sizes for all the folders I can see.
Applications - 5.03 GB
Developer - 2.44 GB
Games - 7.83 GB
Library - 7.21 GB
System - 4.63 GB
User Guides & Information - 4 KB
Users - 16.45 GB
I copied my Desktop Files to my external drive and then deleted it off my computer. That's why it isn't listed anymore.

That adds up to 43.63 GB which winds up being roughly about the size of my backups that Time Machine does. When I do a "Get Info" on my Macintosh HD it says 125.23 GB in size of used space. So if I take the 125.23 GB and subtract the 43.63 GB I get 81.6 GB. Still trying to figure out what file/files add up to that. Using OmniDiskSweeper to figure it out. So if the folders I CAN see add up to 43.63 GB, and the Time Machine backup is around that size, then it probably is backing up everything, right? And if that is true, then that means that more then half of my internal hard drive is being used up by invisible files. Hopefully files that aren't necessary if they get deleted by me erasing my internal drive. That is what I am wanting to do. Get my internal drive backed up, wipe my computer clean, and install MacOS X 10.6. Hopefully then my computer will run more smoothly. With this information I just gave you, what are your thoughts? Think it would be safe to wipe my computer clean once I have those folders I mentioned backed up using Time Machine? Let me know. Thanks.

Reply

Answer 9

jsd2

Level 5

6,245 points

Oct 26, 2010 1:10 AM in response to Brian30

Drive List window
Macintosh HD 23.5 GB free, 125.0 GB used, 148.7 GB total

Macintosh HD window
124.8

Good! The sum of the detectable files is nearly equal to the total used space on the HD, which means that OmniDiskSweeper running from your user account did not encounter any significant permissions restrictions. You should be able to pin down what is taking up the extra space.

As Pondini predicted, your missing space is being taken up by very large files that are somewhere in the private>var folder. The most likely culprits are "runaway" log files in private>var>logs, but there are other places in private>var that can hold abnormal large files. To look further, you could either drill down further with Finder after using Go>Go to Folder>/private, or you could continue to use OmniDiskSweeper. Either way, remember that 1000MB=1GB, so you can basically ignore folder sizes that are reported in MB.

To continue with OmniDiskSweeper, expand its Macintosh window to the right to see more columns, and then "follow the big folders", starting with private on the leftmost column, and clicking in turn on the largest (topmost) folders in the successive columns going rightward. So in my system, the largest item in /private is the 4 GB Sleepimage file in private>var>vm, and my own private>var> log folder is small (8.2 MB=.0082 GB):
.
!http://i55.tinypic.com/30m9120.jpg!

Reply

Answer 10

Brian30 Author

Level 1

5 points

Oct 26, 2010 1:40 AM in response to Pondini

I just wanted to tack this on to my last response to your post. In looking at all the invisible folders that are at the root level of my internal drive, there are 2 that are pretty good in size. The folder called "private" is 79.21 GB in size. The other invisible folder that is good in size is the "usr" folder. That one is 1.38 GB in size. (Got that number by doing a "Get Info" in the Finder) OmniDiskSweeper says its 1.4 GB. And no, I did not forget a letter "e" in that folder name. Its literally called "usr". Anyway, just thought I'd pass this on. I think those 2 folders are whats contributing greatly to my used up hard drive space.

Reply

Answer 11

Brian30 Author

Level 1

5 points

Oct 26, 2010 3:33 AM in response to jsd2

I really want to paste in what my OmniDiskSweeper shows. How are you doing it? Are you just taking a screen shot and then pasting it in the text box here? That's what I'm trying and its not letting me paste it in.

I expanded the Macintosh window. I'll try my best to explain the results.
private-79.2GB --> var-79.2GB --> log-76.7GB --> asl-76.6GB --> asl.

The "asl" folder consists of 4,368 items all ranging in size from 0 bytes all the way up to 61 MB. I don't know if you read any of my replies I left for Pondini, but in one of my recent ones, I stated that there were 2 invisible folders that were taking up the most space. One being the "private" folder-79.22 GB and the other being the "usr" folder-1.38 GB. And yes, that folder name is correct, usr. No "e" in it. You stated that 1000 MB=1 GB. You are really close. I've watched Time Machine start its backup and once it reaches 1024 MB, that's when it changes over to 1 GB. I looked at the size of my Sleepimage and it only came to 2 GB. Not sure why but my "asl" folder is the biggest folder inside my "log" folder. If I could get the image of my OmniDiskSweeper results in here, you could really see in detail then what mine looks like. I asked Pondini this question but I'll ask you too and see what your thought is. I am really wanting to install MacOS X 10.6. Once Time Machine backs up the visible folders on my internal drive (Applications, Developer, Games, Library, System, User Guides And Information and the Users folder), do you think it would be safe to go ahead and wipe my internal drive clean and then install 10.6? I am hoping that all those invisible files are not necessary to keep. And that all my programs will still run fine without all those files. I am really wanting my computer to run better. It takes longer then it should to start up and shut down. Let me know. Thanks.

Reply

Answer 12

jsd2

Level 5

6,245 points

Oct 26, 2010 6:14 AM in response to Brian30

I really want to paste in what my OmniDiskSweeper shows. How are you doing it?

You have to upload the image to a separate online host, then reference it. See Ali Brown's post [here|http://discussions.info.apple.com/thread.jspa?messageID=12100919&#1210091 9].

My last image is actually located here:
http://i55.tinypic.com/30m9120.jpg

and to have it appear in my post, I typed this line:
\!http://i55.tinypic.com/30m9120.jpg!

Once Time Machine backs up the visible folders on my internal drive
(Applications, Developer, Games, Library, System, User Guides And Information
and the Users folder), do you think it would be safe to go ahead and wipe my
internal drive clean and then install 10.6? I am hoping that all those invisible files
are not necessary to keep.

The issue isn't the invisibility - all Macs systems have invisible files and folders which are vital parts of OS X, and which are present in the Time Machine backups as well. The /private folder and the /usr folder you mentioned are among these. Your Mac won't run without those invisible items that are supposed to be there.

The problem in your situation is probably that some process has encountered an error condition which is causing it to write huge numbers of messages to log files, taking up a lot of disk space, and also slowing down the entire computer. These log files are not themselves invisible; the reason you can't see them is that they are stored in a root level folder (/private) that is invisible. We don't know at this point if the process with the error condition is still active, or whether it is no longer generating those excess log files. If it is still active, then even if the current large logs are deleted, new ones will be regenerated and the computer will still be slow. And even if you instead completely wipe the disk and restore everything from Time Machine, the runaway process could get restored as well.

I would look further to try to see what has generated those log files. Open Console.app (in /Applications/Utilities) and if it does not crash, look at the messages and the system log for frequent recurrent entries.

Reply

Answer 13

Pondini

Level 8

38,863 points

Oct 26, 2010 7:28 AM in response to Brian30

Brian30 wrote:
. . .
private-79.2GB --> var-79.2GB --> log-76.7GB --> asl-76.6GB --> asl.

asl is used by the system logging facility, and it occasionally goes nuts. Delete all the date-named files inside it (you'll have to enter your Admin password) and empty the trash.

A few things can cause that; some sort of system problem that sends zillions of messages to your logs; some failure of the logging facility itself; and, eventually, if the built-in maintenance scripts can't run for a very long time.

Check the size of your system.log and the series of system.log.0.bz2, etc. files. If any of them are over about 100 mb, there may be problems. If so, follow jds2's advice about examining them.

Do you power your Mac off every night? If so, the maintenance scripts that are scheduled between 3:15 am and 5:30 am local time can't run. There are ways to run them manually, but for the moment just keep it running (not sleeping) overnight, for roughly one night per month. When you upgrade to Snow Leopard, you can let it sleep overnight, and the scripts will run soon after it wakes up.

Message was edited by: Pondini

Reply

Answer 14

Brian30 Author

Level 1

5 points

Oct 26, 2010 10:13 PM in response to jsd2

I am attempting to load in the images I took of OmniDiskSweeper results from my computer. Hopefully you will be able to see it ok. I took 2 pictures. Here they are.

!http://i200.photobucket.com/albums/aa125/BrianG27/Computer%20Information/Comput er1.png!

!http://i200.photobucket.com/albums/aa125/BrianG27/Computer%20Information/Comput er2.png!

Pondini suggested that I go into the "asl" folder that's inside the "log" folder and delete all the date named files. The image you see here was taken BEFORE I deleted those files. I have since done that.

So, are you saying I should also backup the /private and /usr folders as well just in case I should need them again? When I do erase my computer's internal drive and install MacOSX 10.6, I would think that it would re-create any invisible folders that it needs in order for my computer to run. Is that not true?

The process with the error condition is still active it seems. After I deleted most of the files that were in the "asl" folder, there were a couple that couldn't be deleted because they were in use. I restarted my computer and then was able to delete those couple that were in the trash still. I then went back into the "asl" folder, and yes, more had been created. 5 more to be exact. Just to check it out, I did the same thing again. Deleted those 5 files and then restarted my computer. Went into the "asl" folder and there were 5 new files in there again. So whatever is generating them, is still active. Once I get my computer set back up on 10.6, I would only "restore" files that I want. Maybe a game or some music files. Or a few pictures. I would not restore any system files. So, I'm hoping that by doing this, I can fix what is wrong. I will admit however, that if my computer had major issues and I had to restore my computer from my Time Machine backup, this process would most likely start up again.

Just as a little FYI. Since deleting the date named files in the "asl" folder, my computer now has 100.11 GB available. And the "Used" space is only 48.62 GB. Quite the difference from what I had originally said I had for "Used" space.

I also took a picture of my Console windows. One is "System.log" and the other is "All Messages". Maybe seeing this will help you to help me figure out what's going on.

!http://i200.photobucket.com/albums/aa125/BrianG27/Computer%20Information/Consol emessages.png!

Reply

Answer 15

Brian30 Author

Level 1

5 points

Oct 26, 2010 10:47 PM in response to Pondini

I have gone into the "asl" folder and deleted all the date-named files and then emptied the trash. There were a couple that wouldn't delete because it said they were in use. I restarted my computer and I was then able to delete those couple. I then went back to my "asl" folder and 5 new date-named files had been created. I deleted those 5 and restarted my computer a second time in order to delete the ones that were "in use". I then checked that folder again and 5 new files had been created again. So, whatever process that is creating those files is still active.

Just an FYI. Since deleting the date-named files (there were over 4,000 of them), my computer now has 100.11 GB of free space out of a capacity of 148.73. That makes the "Used" space 48.62 GB in size. Definitely a major difference from the sizes I had originally quoted, don't you think?!

I have posted a picture here of my Console windows. One of them shows the size of my system.log. Says its 5.5 MB in size.

!http://i200.photobucket.com/albums/aa125/BrianG27/Computer%20Information/Consol emessages.png!

Yes, I do shut down (power off) my Mac every night. Unless I am allowing it to do a Time Machine backup, then I leave it on. Or if I am downloading a huge file from online but that very rarely happens. I did not realize there were maintenance scripts that ran between 3:15 AM and 5:30 AM. This is the first time I have heard of that. Maybe I could mark it in my computers calendar to leave my computer on overnight on the 1st of every month. I could even set up a reminder for it too. But just out of curiosity, how do you run them manually? I will try to remember to make that note in my calendar once I have upgraded to MacOSX 10.6.

Maybe by looking over my Console windows, you will be able to help me figure out what is going. Really anxious to get my computer upgraded to MacOS 10.6.

Reply