I am copying the contents of a nearly full old LaCie 500G firewire drive formatted in Mac OS Extended onto a new San Disk 2T SSD external formatted in ExFAT and so far, with about 300G copied from the old drive it has taken about 1.3T on the SSD! Is the ExFAT format really that much less efficient in storage or might there be some other problem here. Should (and can) I reformat the SSD drive to Mac OS Extended?
iMac 27", macOS 10.14
If you have no plans to share that SanDisk SSD with either Linux or Windows, then for efficiency, it should be formatted as HFS+ extended/Journaled with GUID partition scheme. Simply use Disk Utility to erase the SanDisk SSD, and start over with the data transfer.
If you have no plans to share that SanDisk SSD with either Linux or Windows, then for efficiency, it should be formatted as HFS+ extended/Journaled with GUID partition scheme. Simply use Disk Utility to erase the SanDisk SSD, and start over with the data transfer.
Yes you should reformat to HFS+ (aka Mac OS Extended (Journeled)) as suggested by VikingOSX
HFS+ has a 4096 byte allocation unit size.
If the exFAT drive has a much larger allocation unit (say 64K), then a small file that would fit in 4K, will be given 64K whether it needs it or not. And all files will be rounded up to the next 64K boundary when storage is allocated.
NOTE: I DO NOT KNOW what allocation unit size the exFAT was formatted with, so consider this total speculation.
Another possibility (although a bit remote) is if you have lots of hardlinks and symbolic links between files on the LiCie. I do not think exFAT supports linking multiple file names to the same physical file, so a copy process would duplicate the file instead of creating links to the single physical copy.
Think of file links as your cell phone contacts list. You have a copy of your friend's phone number, but other people have your friend's phone number as well. So while there are multiple pointers to your friend, your friend is still just a single person, and has not been cloned.
That is what a file system link is, a directory entry that points to the physical copy of the storage, and there can be multiple links point to the same file. But the file system has to support file system links and I'm not sure exFAT has that support.
Again, I DO NOT KNOW if you have been using hardlinks or symbolic links on your LiCie (or if software you have been using has been taking advantage of file system links), so this is just total speculation on why your 300GB has grown to 1.3TB
Thanks BobHarris, that's what I did. I did not know about about that kind of symbolic link structure. Its interesting. I wonder how the system handles it if you delete the file that happens to be the one with the physical copy but don't want to delete the file in another place where it is only represented by the link. I think this is more likely to have been the source of the problem since the disk allocation approximately quadrupled and the unit allocation difference, as far as I can see should not have made anything like that much difference. There are in fact many duplicate copies of files in what I backed up.
Deleting files.
There are 2 kinds of file system links. Hardlinks and Symbolic links
A Symbolic Link (sometimes called a symlink) is from one perspective similar to a macOS Alias (there are a differences, but for this discussion that is its closest relative on the Mac). The symbolic link is a file that contains file path information to the real file. That is to say if you have /Applications/Utilities/Terminal.app as the file, you might create a symbolic link in a different directory that stores the string "/Applications/Utilities/Terminal.app" as the contents of the symbolic link, and the file system knows to read that string and open the real file. The file system knows to look for that string because the file type is Symbolic Link (vs Regular File, vs Directory, vs Block Device, vs Raw Device, vs FIFO (aka a named pipe), etc...).
If the real file is deleted, the symbolic link is orphaned, in that there is no longer a real file to be opened. But if you later create a new file at that location with the exact same file name, the symbolic link will again find a file to open. This is how a macOS Alias behaves as well.
NOTE: A symbolic link is a low level file system abstraction. A macOS Alias is a GUI framework abstraction. Generally speaking a symbolic link is accessible to every programs and process running. An Alias is really only available to GUI applications that use the GUI framework to access files.
A Hardlink is a 2nd (3rd, 4th, ...) directory entry that contains a file name (does not need to be the same name as the other file), and an internal file system file identifier. In the Unix world this is called the "inode" number. The inode is how the file system finds the metadata for a file, where the metadata is things like time stamps, file size, owner, group, permissions, and what blocks of storage contain the data for the file.
Anytime a file system hardlink is created, the file system increments a reference count on the file. In the Unix world this is called the "nlink" count.
When a file is deleted, the file system removes the filename from the specified directory and decremented the nlink value. If the nlink value goes to zero after it is decremented, then the file system will mark the file for deletion. If the nlink count is greater than zero, then it means there is 1 or more other hardlinks in some directory within the file system that is pointing at the file's inode.
I say marked for deletion, because if the file is open and by macOS, then the file will not be deleted until it is closed for the last time (as in it could be concurrently open by several processes).
My day jobs is as a Unix file system developer, which I've done for several not-Apple Fortune 500 companies.
Thanks, that's what I did and problem solved.
You are welcome.
file format efficiency
