du, df and where the **** is my space gone

I see it's very strange.

I am wondering : how feasible is it to clone my /dev/disk0s2 (not a binary copy of course) to an external HDD, format the partition on the SSD and then copy it back from the HDD to the SSD ? Any experience.

It may work but I'm not sure. I don't have such experience.

Most similar I did was setting up new mac with Time Machine data of other Mac, and I didn't compare the disk usage.

By the way, have you tried 'verify disk' or 'repair disk' with Disk Utility?

( It is better to be done in Recovery mode. OS X: About OS X Recovery - Apple Support)

The mismatch may be due to incorrect data in the file system.

Some tips for 'du' and 'df'

For 'du',

You can try, sudo du -d 1 -xch /

In this case, the directories and files starting with a period are also reported.

-d 1 — report one level below.

-x — don't traverse to other mounted volumes.

For 'df',

You don't need 'sudo'.

You can give '/' as argument to show the data of boot volume only. ( like my last post )

You better know,

1. SSD labeled as 256GB often means 256*1000*1000*1000, but the unit GiB used in the output from 'df -h' or 'df -g' is 1024*1024*1024 B. This makes 256GB = 238.4 GiB. The 'du' use GB in meaning of GiB. ( check 'man df' and 'man du' ).
2. In contrast, the sizes shown on File Info of the volume in Finder are in GB (= 1000^3 )
3. EFI and recovery partitions occupy nearly 0.9 GB ( on my mac, may vary on other models ). They are hidden from usual operations, but it can see with 'diskutil list' command in Terminal.
4. 'du' calculates the total of files those can be accessible. However, some system files are hidden from normal user operations. In other words, you must run the command with administrator privilege, e.g. 'sudo du -s /', to get the total size of all files on the system.
5. The numbers reported on du and df don't rely on the spotlight database.

Hope these info will be a help.

OK, I see.

I did sudo du -csh * from / directory.

In the shells ( command line interface of UNIX system ), ‘*’ matches all file names, but which starting with ‘.’ are exception.

You need to specify the files/pattern specifically. Try sudo du -csh /.??* for those you did not get with above command.

This will give the disk usage of directories with the name starting with ‘.’ directly under root directory.

Below shows some examples.

/.MobileBackups — Time Machine backups ( local snapshots )

/.Spotlight-V100 — Spotlight database

/.fseventsd — Used by File System Events service, which notifies the updates of files to programs/services, such as spotlight.

Also, some small portion of the partition is used for file system itself. As it does not belong to any file, du cannot see it.

( I have no specific data about how much of the disk space is used by file system. but you can get some idea by comparing the free space and partition size on empty volume )

I have no more idea what makes the difference on your volume.

Are you sure you did run ‘sudo du -csh *’ in root ( / ) directory?

Here is what I get on my macbook pro with 512GB SSD running latest El Capitan ( To be precise, the numbers are shown in 513-byte blocks )

mac:/ 268 $ sudo du -shx / 2>/dev/null

Password:

236G /

mac:/ 269 $ df /

Filesystem Size Used Avail Capacity iused ifree %iused Mounted on

/dev/disk1 465Gi 254Gi 211Gi 55% 66531527 55306071 55% /

In this case, 254 GiB is used on 465 GiB partition with 236 GiB for files and directories.

I think the difference ( 18 GiB ) is compensation for file system ( and due to some problems as I run while both Time Machine and Spotlight are active )

cd /

sudo du -csh *

Is not going to find any of the /.name files and directories in the root file system. To_Mi has already told you how to get those included in your du run.

You should also include -x so that you do not try to index external drives. Then you would not get errors from your NAS devices.

There is a very good chance 'du' is ONLY telling you about the data used by all your files, and NOT telling you about all the file system infrastructure needed to keep track of your file and its data. This would be things like the file system metadata to keep track of the owner, group, permissions, file time stamps, etc..., and all the metadata needed to keep track of what physical storage blocks are part of your file and in what order those blocks are to be accessed. When you have millions of files on a file system (129 millions on my root file system), that metadata starts to add up.

df is a file system level report on the resources available and used. It maintains these few values and updates them then storage is allocated and deleted. The allocation numbers do not care if the storage is going to use data or to file system metadata, it just needs to know if it has storage to give when there is an allocation request.

Reindex spotlight.

Did you do just "sudo du" ?

By default, du only search for the files and directories under the current directory.