how do i securely erase free space on Catalina

Are you on an SSD? If so, the basic device I/O operations are wildly different from those of a hard disk drive (HDD), and the traditional assumptions based around how hard disk drive erasures work are utterly inapplicable to SSD storage.

Among the differences...

...SSD sectors are erased very shortly after the files are deleted, and must be erased before the storage is re-used, and SSDs work hard to make that erasure happen quickly. This is what TRIM expedites, and macOS supports TRIM.

...SSD sectors cannot be overwritten, due to wear leveling. Among other fundamental differences.

...You can’t write to the same physical sector on an SSD, as sectors are remapped on each deletion, and each rewrite. This due to wear leveling. (HDDs have an earlier form of this, which makes overwriting problematic.)

...The multiple-pattern-overwrite stuff dates back to floppy disks and to 1980s-era hard disk drives, which all had really sloppy head positioning. Modern HDDs, and particularly SSDs, work very differently here.

Encrypt your storage. Storage encryption combined with a secure password protects all of your data, including against any data that might remain on a failing or failed or replaced storage device; including any remaining data in a hard disk free space.

The Secure Erase operation is the analog to a pattern overwrite in a hard disk.

But whole-volume encryption is the preferred approach, and for various reasons.

If you’re even asking about free space erasure, your data storage is likely not encrypted, and you’ll want to fix that now.

Applying old lessons and old remediations against new situations and different platforms doesn’t always end well.

Figured out that if you disable System Integrity Protection (probably not a great idea) you can mount the volume as read/write, and then you can run the diskutil command. You get a fun new error though:

Error with secure disk erase: Secure erase by writing a run of bytes to an APFS Volume makes no sense due to its possibly-unbounded size (-69489)

So, it seems that this functionality is broken by APFS. Hopefully Apple is working on another way to make this work, but in the meantime it just means my VM hard drives will take up more space on my physical machine I guess.

SSD storage that wasn’t been written isn’t allocated.

Storage that has been written is allocated, and the only way to release that is with TRIM.

Overwrites and erasures are futile with SSDs.

If the VM is somehow re-mapping guest storage—beyond the remapping that SSDs implement for wear leveling, and the reason why overwrites don’t and can’t and won’t work, and why TRIM now exists—please check with the VM support folks.

TRIM is the low-level command that does what you want, and you will need VMware to issue some sort of TRIM request to release the storage.

Overwriting is an HDD concept.

Overwriting does not do what you want on SSD, save as a (bad) way to send a TRIM at the storage, and to further wear the storage.

Every write on SSD goes to a different hunk of storage. Every write. Not into the same storage sector as the data was last written into. You can’t overwrite free space on an SSD, as that’s already happened secondary to remapping and the TRIM request that was implicit when the old storage was released.

SSD storage is virtualized, for wear leveling. Every write gets reallocated storage elsewhere on the SSD retrieved from the free list, while the old storage gets queued for erasure and to be re-added onto the free list. Flash doesn’t have an overwrite, it has a write, followed by an erase and a rewrite, and the erase is comparatively slow, and that and wear leveling leads to a very dynamic no-overwrites I/O scheme.

Contact VMware Support, and ask them for assistance with releasing the storage, either some existing VMware command to re-pack the virtual volumes (and TRIM what gets released) or by adding some new feature. If VMware fully virtualizes the storage, then they could conceivably re-pack the contents of the virtual volume, and then release the host-visible storage, which would then get TRIM’d.

Did some more digging, and confirmed with a VMware community thread (https://communities.vmware.com/thread/468349) that it is possible to force the storage to appear as an SSD to the guest operating system. Did that, and can't get MacOS to enable TRIM support on the device, even with the built-in trimforce utility.

Then, started thinking some more and realized that since all I needed to do was write zeros to the freespace, I could do this:

cat /dev/zero > wipefile; rm wipefile

Basically just writes zeroes to a file until the drive fills up, which causes the write command to fail, and then deletes that file. VMware disk tools then correctly interpret the empty space as having been zeroed, and will trim the backing VMDK accordingly.

So, turns out I don't need the secure erase functionality for my use case. Not sure if helps the original poster though.

BobHarris wrote:

You can even use the dd command to do it.

Yes, this did the trick for me. Thanks! Here's a good article about how to proceed. I wasn't convinced the mkfile command was actually writing data, as the Finder reported just 4K for the number of bytes "on disk" (the value in parentheses for the Size reported in Finder's Get Info dialog).

The dd command, on the other hand, reported the expected number of bytes for both the Size and "on disk" size, and I confirmed the file was actually filled with zeros by opening it with a hex editor. Bada bing! 👍

It can take a while, however, for the file to be written in its entirety for really large files (a TB or more) - just like the utilities which overwrite free space.

Just noticed in my TechToolPro 11 App, it has Wipe Data Securely for a File, A Folder, a Volume, or Just Free Space.

Erase free space is just a program that writes a single file of zeros until the disk is full and there is no more free space, then it deletes the file. Like magic all the free space is filled with zeros.

Anyone can write this kind of stupid program.

You can even use the dd command to do it. The trick is making sure the file gets deleted after all the free space on the disk has been consumed. macOS most likely had boot code that would do that in the event of a crash.

This is a stupid script to create a file of zeros. I do not know if it is sufficiently robust. I'm sure you are in trouble if the Mac crashes and does not get to run the rm command. Or kill -9 is used, or any number of other ways the rm command can be prevented from running.

#!/bin/bash
ZERO_FILE=/tmp/file.of.zeros
trap "rm -f $ZERO_FILE" EXIT
dd if=/dev/zero of=$ZERO_FILE bs=4096

If nothing else you can fill the drive with garbage & duplicates of garbage, the n trash them & empty the trash.

You are correct - there is still a diskutil command with a freespace option. If I run:

sudo diskutil secureErase freespace 1 /Volumes/MacHD

I get:

Erasing freespace only works on mounted and writable volumes

This appears to have something to do with the way that volume writes are handled with APFS volumes, which is new to Catalina. Presumably, if I could get the volume remounted in a writable state then I could use the existing secure erase option.

To that end, I've tried a few things. I rebooted in Safe Mode, which didn't work. I tried the following command based on an Adobe forum post that seemed similar here:

sudo mount -uw /

That fails with:

mount_apfs: volume could not be mounted: Operation not permitted
mount: / failed with 77

Could you give some details or site some references about this problem? There is still an entry in the man page for "diskutil" that talks about doing secureErase option that has a "freespace" argument in the Catalina version of man pages. Other than the SSD issues that MrHoffman talks about, this seems like something that would be something some might want to do occasionally to real hard drives and even Fusion drives I would think (or do they have the same behavior as SSD drives)?

And just want to make certain also about the OP wanting to erase only the free space or the entire volume/drive? As well as are they talking about SSD or Fusion or normal spinning hard disks?

Thanks...

Shotster wrote:

I'm selling my iMac with Fusion Drive, and having read this thread, I'm sill unclear about whether it's a good idea to wipe the drive or not. I can't do it with Disk Utility or from the command line. It would give me peace of mind to wipe the drive before getting rid of the machine; or is my concern unfounded?

If you followed the Apple install-time and upgrade-time defaults and have encrypted your storage using File Vault, you’re fine. File Vault uses a robust password to encrypt the storage, even if your own chosen password is garbage. Overwrite the File Vault password, and the encrypted data becomes entirely unrecoverable. This given the “best” approach involves many years’ effort brute-forcing the generated password used by File Vault. (Your password is a password to the generated password, and not the key to the storage encryption.) The T2-class Macs do far better here, using the T2 to encrypt the storage.

If you had previously decided to “YOLO” with your data and not have File Vault enabled and now choose to become concerned about the data, then boot Recovery, use Disk Utility to erase your storage, and then re-install macOS, and that will probably work well enough for your needs. One-pass overwrite will thwart the sorts of access attempts most of us are concerned about.

If you’ve classified or sensitive data here, then remove the storage from the iMac, and physically destroy it. Fusion drives aren’t worth all that much. For best re-sale value, replace it with an HDD or SSD.

Shotster wrote:

MrHoffman wrote:

If you’ve classified or sensitive data here, then remove the storage from the iMac, and physically destroy it. Fusion drives aren’t worth all that much. For best re-sale value, replace it with an HDD or SSD.

Say what??? A HDD is better than a Fusion Drive?

Replace the now-gone and now-scrapped Fusion drive with something to help the resale value as otherwise the buyer will have ro do that. I’d replace the Fusion drive with either an HDD (cheaper, slower) or SSD (more expensive, faster). Not with a Fusion drive, as those will tend to be specialized purchases and somewhat more involved to configure. An SSD can be gotten from OWC for instance and for what will probably be a purchase price competitive with an Apple Fusion drive.

SSD erasure is wildly different from HDD erasure. Files on SSD are erased on delete, with TRIM and with Secure Erase. On HDD, not so much. With a Fusion drive, I’d expect to split and then overwrite the HDD component of the Fusion drive, then rebuild the Fusion drive for the buyer. If you don’t “just” replace with an HDD or SSD, and scrap the Fusion drive. This if the Fusion drive contents were not previously encrypted.

How to fix a split Fusion Drive - Apple Support

How to erase a disk for Mac - Apple Support

What other explanation can there be? Data compression, and copy on write.

Some related reading:

http://osxdaily.com/2016/08/30/erase-disk-command-line-mac/

dd if=/dev/random {rest of command}

Files filled with random data (and thus also with encrypted data) do not compress.

I'm trying to figure this out as well. I have a couple of VMs that I use for testing and the only way to shrink the virtual drive files (recover unused space) is to write zeros to the freespace. I used to be able to do this from the command line with diskutil, but that doesn't seem to work now as it seems APFS volumes are non-writable. Tried mounting the volume as RW, but that gives me errors as well.