APFS vs. exfat vs HFS+ space consumption (very) different

Hi,


I know that there are (a lot of) differences between the three filesystems and all have advantages and disadvantages. Or are recommended e.g. for SSD or HDD etc. Thats out of question.


Recently we synced data from a SD card to two identical 2TB harddisks and got the error, that one harddisk is full, while the other still has +- 400GB free space.


I checkt with all tools I know: Disk utility, repair, fsck, du, rsync both directions, find and both disk hold the same files, the subfolders are +- a few bytes now and than identically.


The difference is: The full HDD is APFS forrmatted, the with +-20% free space is exfat formatted.


The Files copied are multi-GB video or audio files, so "blocksize-wasting" is not seen.


So APFS is doing something veery strange here, but what?


So I took two other HDDs of 2TB and copied 948GB "testdata" to different filesystems. The result:


  • exfat: 948,55 GB used / 1,05 TB free
  • Mac OS Extended Journaled: 949,36 GB used / 1,05 TB free
  • APFS: 1,04 TB used, 964,63 free.


All disks/filesystems had been formatted with the defaults and no other options or features had been enabled or configured by me.


Has anyone an explanation where the space is gone, how dose APFS fills up that disk?


Thanks for suggestions and related posts/doc. Regards . Götz


Posted on Mar 31, 2022 5:34 AM

Reply

Similar questions

11 replies

Apr 1, 2022 10:45 AM in response to goetz.reinicke

Your findings are interesting and I suspect there may be a number of possible explanations for the difference in the APFS size readings vs HFS+ and ExFAT. Unfortunately APFS is not all that well documented, at least not for us mere mortals. What has become apparent, however, is that under APFS disk usage is not calculated the same way it is on HFS+ and other formats. Here's an article about APFS Disk Free Space that may be useful.


One of the differences between APFS and HFS+ is that APFS allocates file metadata alongside the file data, meaning that the metadata occupies space on the disk that would otherwise be free space (including any overhead where the actual metadata is smaller than the minimum space allocated for a file, block or other unit.) On HFS+ file metadata space is preallocated in a single area of the disk.


I'd be interested to see what Disk Utility says about the usage on each of the disks you are testing (the devices as well as the volumes on each).

Mar 31, 2022 11:44 AM in response to goetz.reinicke

goetz.reinicke wrote:

Has anyone an explanation where the space is gone, how dose APFS fills up that disk?

A quick review of storage issues posted by users here is the forums should tell you that APFS was not designed for space efficiency. Lots of effort has gone into making copies faster and support for snapshots, cryptographically sealed volumes, dynamically sized volumes, etc. But storage space? That costs money.

Mar 31, 2022 7:32 AM in response to goetz.reinicke

My only comment is we abandoned cards because they never worked right. Always flakey in one way or another ... never reliable.


I bought a pair of inexpensive 250GB external SSDs to replace all our card uses. One of the best decisions I ever made. Have had zero problems since.


The only use I have for cards or flash drives now is for the installation of macOS.

Mar 31, 2022 1:31 PM in response to goetz.reinicke

goetz.reinicke wrote:

Hi, i searched half a day and did not find any hint on the space consuming... just about the features you mentioned.

This is tech support forum, not a technical forum. Search for terms like "other storage" or "out of disk space". That is how end users typically encounter these problems. They have no idea what APFS is.


But you do have to be a little careful. Many of these reports are based on using APFS as a boot drive, with Time Machine. As external storage, APFS is going to behave much differently. But it is entirely reasonable that the factors that will cause a boot drive to lose hundreds gigabytes could cause the small differences that you have observed in your tests.

And to be honest, I'd love to see some "fact" sheet or rule of thumb by apple.

Apple doesn't publish any documentation at that level of detail. Generally there is marketing materials and developer materials. There is rarely anything in-between. And developer materials consist entirely of how to use APIs, never about internal design or usage characteristics.


But ironically enough, Apple has published more documentation at this level of detail for APFS than for just about anything else. So while there is virtually nothing, objectively speaking, APFS is exceptionally well-documented, relatively speaking.


Here is some older FAQ documentation.

Here is the current, official documentation.

Here is some sample code.

And here is the official reference documentation.


That last one might be the only one that has any content that might be specifically applicable to your question. However, I admit that it isn't very accessible.


If you like it, make sure to download a copy and save it. The few times that Apple has ever published the kind of comprehensive documentation you are talking about, they have almost always taken down eventually. They are still available if you know where to look for them, but I can't reference them here anymore because they would be copyright violations. For example, there was a great document on how to use autoFS years ago.

As mentioned: 20 to 25% "ghost space" is way to much for any security or new layout etc ....

That is a subjective assessment. As I mentioned above, Apple sells SSDs at 100-300% premiums over off-the-shelf M.2. They are faster than off-the-shelf M.2, but you are paying for speed, not efficiency.

So can you share some posts here with me please? Thanks.

I've given you all I can find.

Apr 1, 2022 12:11 AM in response to etresoft

Hi and thanks for the detailed feedback. I'll cross check if I can find some information for my case.


But again I have to point something out:


I talk about external non apple harddisk, 2TB each same model, same case, same USB3 connection .... and the same files on it.... on a macbook i9.


Plain formatted with one partition, non encryption etc - just different Filesystems.


Not SSD, not SD. - Spindels!


And it is not a "subjective assessment" if I have 450 GB free on the extfat disk vs. the APFS disk which has 1 GB free. And of course security, snapshotting etc would consume space I take that in account, but if you dont use such features, the space should be usable for data. But may be I understand your "subjective assessment" wrong.


For that case I dont have the screenshots at hand. But on the screenshots you can see, that APFS is using obviously more space than exfat.


Regards . Götz

Apr 1, 2022 5:36 AM in response to goetz.reinicke

goetz.reinicke wrote:

Not SSD, not SD. - Spindels!

The physical mechanism is not relevant.

And it is not a "subjective assessment" if I have 450 GB free on the extfat disk vs. the APFS disk which has 1 GB free. And of course security, snapshotting etc would consume space I take that in account, but if you dont use such features, the space should be usable for data. But may be I understand your "subjective assessment" wrong.

I was referring to your subjective assessment about “20 to 25% ‘ghost space’ is way to much”. As long as people keep buying, and paying extra for it, then it isn’t a problem. You might think it is wrong, but that’s just your opinion. It’s 2022 now. Public opinion has more value than empty “facts”.


This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

APFS vs. exfat vs HFS+ space consumption (very) different

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.