I have had this problem on multiple occasions since upgrading to Tahoe, most recently when corrupt Calendar cache files would cause it to instantly gobble up 13GB and immediately throw the "out of application memory" error - even when no other app was running. Whilst I don't believe Calendar is the root cause of everyone's problem, it did provide a repeatable instance of the problem and force me to think about the issue.
It is easy to dismiss the problem as "not Apple's fault" and/or "must be some other software causing it", but there are too many users suffering this issue and the only common factor is Tahoe. The chances of all the victims running exactly the same rogue third party softwares is not even worth considering.
So why do some users repeatedly suffer this problem, while others never do?
I think it's down to drive space. Some have larger drives than others. However, there's more…
As has been stated here, the "application memory" in question is swap space (but Apple have dumbed it down to the point where very few understand what the error actually means) and when the drive is getting full (although still fully operational in all other respects), Tahoe is unable to swap memory to the hard drive and depending on usage, can cause this error. Even the worst affected may not see it all the time and some maybe never. Just depends on the size of their drive and how much is already being used. This is the 'user variable'.
While IMO Tahoe should do a better job at dealing with this issue, I believe there is a bigger problem here that has not been addressed, but is pertinent to the issue.
Where has all the drive space gone?
I specced this Mac Mini M4 with a 512 GB drive, based on the fact that for many years I have had that size in the Mac I was using and it remained approx only half full. Let me repeat that. For probably 10 years or more, my drive usage has NOT increased and my computing habits have clearly not changed in that time nor, I can confirm, since.
So when setting up this Mini, I was using about 250GB. Now, having been using Tahoe for some months, it is over 450GBs. Where has that 200GB gone?
That's for me to look into further, but my point with regard to this topic is that I believe the "application memory" problem, while not well handled by Tahoe, is merely the direct result of Tahoe eating drive space and how that impacts on the remaining space (hence the swap availability) and this will vary enormously between users.
It won't just be down to size of the drive. Someone with a small drive may never suffer the problem as their usage means there is still plenty spare, even after a huge chunk having been munched by Tahoe. Conversely, someone with a large drive may suffer the problem because they were already using a large (but acceptable) portion of it, before Tahoe stole enough of the rest to cause the 'application memory' problem and subsequent error - from which in either case it may not have been recoverable.
So I think the big question is - why is Tahoe stealing enormous amounts of drive space and what is it doing with that space and then of course, how can we get it back and stop it from happening again?
Ultimately, IMO, this entire issue is indeed a Tahoe problem.