Like I mentioned earlier, the case sensitive workaround is masking an underlying issue that Apple still needs to address. The good news is that formatting your Time Machine backup drives as case sensitive should have no negative side effects since the disk that Apple uses in the Time Capsule version of their Airport product comes formatted this way. So even when Apple fixes the underlying issue you can continue using your case sensitive formatted Time Machine backup disks.
Note that the format of your Time Machine backup disks do not have to match the format of your Mac's operating system disks. Your Mac can be using any supported format, e.g., APFS, Mac OS Extended (Journaled), etc., and still back up to a Time Machine disk formatted in any of the supported formats that Time Machine works with. By the way, Time Machine backup disks cannot use APFS.
In a perfect world all disks would be formatted case sensitive. It is more secure and logically correct. Unfortunately some legacy operating systems were only case insensitive and programmers took advantage of this sloppiness allowed by those old operating systems. There are some very large applications, for example some of Adobe's, that break if you try to run them on a case sensitive drive. These apps probably have a lot of legacy code in them and the developer doesn't want to clean it up because it's still working. But since you're not running applications on your Time Machine disks, using case sensitivity fro Time Machine is not a problem. Time Machine disks are simply used for storage and all of the disk formats supported by Time Machine are fully "case preserving" which means they will never change the name of a file even if the case insensitive file system doesn't care.
In any case, Time Machine was designed to work with case sensitive and case insensitive formatted drives. There is currently a bug in Time Machine that is masked when case sensitive formatting is used on the backup drive. From reading this forum it sounds like there may be other things like the name of an App Cleaner & Uninstaller file that is triggering the bug. But deleting that app is not a fix either, it's simply avoiding a condition that causes Time Machine to fail when it should either tolerate the file naming error or flag it. Time Machine should be resilient to such user/app induced anomalies and not fail the entire backup process.
Finally, others have mentioned deleting the "inprogress" container from the Time Machine backup disk. At least on my machine, I am not able to delete this container. It gets marked for deletion and moved to the trash bin, but I cannot remove it from the trash bin even with admin privileges. This container stays in the trash bin forever until I reformat the Time Machine backup disk, which means it is still on the Time Machine disk. Not sure if this matters, but it's another piece of information for the Apple engineers to consider.