Time Machine stuck "Preparing backup," out of obvious fixes

Aaaaand, of course, nothing invites a machine to crapout better than having no working backups.

Monday morning, the Mini gets in a boot loop after about an eighth inch of progress bar. Hardware diagnostics run fine. Recovery mode Disk Utility health check diagnoses the drive with scary sounding APFS structures corruptions. Godforbid Disk Utility should have enough brains to know how to fix them (that's only its job, after all), and Alsoft still doesn't have an APFS capable DiskWarrior, so that's the name of that tune. Thank you so much for APFS, Apple. (I'm thinking that earlier versions of these corruptions may have been the reason for my inability to get Time Machine operational all this time.)

Attempts to "restore" the contents of the boot drive to an external so I can reformat the external and migrate the files back later fail uniformly at the very end of the process (of course) with "APFS inverter failed to invert the volume -- Invalid argument." (Thank you, thank you, Apple.)

So I put the Mini in Transfer Disk Mode, and hooked it and the external up to another computer, and prepare to use Carbon Copy Cloner to save the contents for later migration. But wait -- the Mini has an A2 chip, so my data can be safely secured from myself! When I attempt to mount the Mini's internal, I am asked to "Enter a password to unlock the disk blahblah." But there is no way to enter anything in the password box! Absolute icing on the cake.

Since I don't expect to attract any advice for this problem in a thread whose subject is Time Machine, I have posted it separately here. But I thought I'd provide some closure to this Time Machine issue for posterity.

Yeah, I'm aware of the heat issue with kernel_task -- it hits my MBP every time ambient goes over 80°, and I used to use it on outside jobsites in AZ a lot. But that hits all the cores, plus I've never seen it on the Mini.

After restarting after removing the nonexistent folder, kernel_task seems to have stopped spinning. Yay!

Again, it's stuck at "Preparing backup." Now I am seeing the following errors in the log (I've seen them before, too). This whole "file:///" issue may be the key to the problem. I have no idea what it's trying to do here. There is a file .com.apple.timemachine.supported at the root of Tardis, but it's zero-length, and other discussions indicate it should be, so I don't think it's involved. There's nothing else in the hierarchy that looks like it would be involved in whatever TM is trying and failing to fetch.

The only "cleaning" SW I use is when Parallels Toolbox wakes up and offers to remove nGB of old cache/logs from my system, which is a pretty respectable utility -- but in fact until I fixed it last night, my PT on this machine had been expired for years, so that's not it. The only AV I use is Malwarebytes, run only manually by hand every few months, and I've never seen it find anything but adware (and practically never anything on this machine).

The log is interesting. I discovered one of my "exclude" folders no longer existed. (The NSURLIsVolumeKey error seems to be a second symptom of the same nonexistent folder.) I removed it from my exclude list, whereupon the TM backup said "stopping." I restarted it from the menu bar icon, but the System Preferences panel never went to the "preparing" screen, yet the menu bar icon thinks a backup is running.

I don't know what the "sizing operation failed" problem is. Those folders exist, are perfectly accessible, and show perfectly good sizes in the Finder. I tried removing and reinserting them into the exclusion list, and now it just repeats the error message three times in the log for each of them instead of twice, while stuck at "Next backup: one second."

Also, I have no idea what all the "file:///" cruft in the log is. I have nothing like that in my exclude list (image).

Finally, I left the machine alone until the next time TM said a backup was due; it's now past that, and it's still showing the same screen, nothing new in the log, and the menu bar icon still thinks TM is running. (Woops, it just jumped the next backup time one hour, but didn't do anything else.)

The log from before I removed the nonexistent folder from my exclude list is in additional text (1).

Additional text (2) is the log from after that.

I'd still also like to know why kernel_task is eating up 100% of a core on an otherwise idle machine.

(Did this system blackhole my additional text entries? I can't find any way to access them.)

I ran DU First Aid on the TM drive back in the OP. Since I had completely removed the Backups.backupdb folder in order to start fresh, there was no multilinked anything remaining on the drive and the operation was quite prompt. On that drive there are only three sparsebundles of backups from three other household machines, plus a bunch of invisible cruft belonging to the system. The backup drive is in Spotlight privacy.

My boot drive is 130/500GB full, a pittance in today's terms. Of that, /Users takes under 13GB, and /Developer runs 2.5GB, the rest is all vanilla macOS. However, your question reminded me that there is also an external drive for large files, some of which does get backed up as well, and I had never yet checked that. DU loved it long time, but DiskWarrior said it was replacing the directory because the old one was too damaged to compare (no data was lost). For kicks, I also ran DW over the backup drive (since there is no multilinking on it yet) -- DW reported trivial corrections similar to those reported almost any time you run it.

I restarted TM, got the same spate of "file:///" errors, and it's been "Preparing backup" for about five minutes now. I'll let it run and see if that fixes it, I'm thinking my odds are only maybe 20%.

I've run DU First Aid on every volume on that system at least once in the past 24 hours, and DW on the two externals (the internal is APFS). They are now all as clean as they can be.

Both externals are RAID-1 (mirror) volumes that well predate this Mini; I expect the archival volume structures collected some damage during power outages or whatever, or maybe even due to some bug in a version of the RAID driver (SoftRAID). I'm still a bit fuzzled about how DU could say it was perfect, while DW said it was beyond use -- what kind of damage does that? In any event, I'm pretty confident the hardware is solid.

DiskWarrior isn't a hardware thing. It's just a smarter and more aggressive fsck-type utility. I've been using it for well over 20 years to repair filesystem problems that DU never even notices, and to reconstruct volume directories that DU claims are hopelessly unreadable. I trust it implicitly.

I have no reason to even suspect anything is wrong with any of my hardware. Startup-D gives a clean bill of health, and there's really no reason for the SSD to be going sour. (And if it happens again, I can just replace it, once I have a recent TM backup).

I am currently using another (working) A2 device to clone my drive from the Mini to an external so I will have something to run File Migration from, then I am going to nuke the internal and reinstall a fresh Mojave.

Yes, you bet I am fixated on APFS. When I get loads of error screens from (useless) DU saying stuff like:

error: Cross Check: External physical extend (xxx + 1) has kind APFS_KIND_UPDATE 
      but was not referenced previously {several of these}
Snapshot is invalid
APFS inverter failed to invert the volume -- invalid argument

...I don't have to run DNA to ID the probable culprit. I repaired and maintained Macs for 35 years without ever having to "invert a volume," whatever the heck that is, and it looks like something I could easily have gone without.

If DU were up to its job, Alsoft wouldn't be in business. But there's a reason they've been profitable for decades.