low memory

Does this pertain to High Sierra OS where this posting in placed ?? Open Activity Monitor >> View >> View All processes. Look in CPU Tab and identify any process or application that is using unusually large amounts of CPU. Do the same in the Memory Tab. Using this method may help identify a Run Away process or application that is malfunctioning.

If this pertains to Big Sur as you profile shows running macOS 11.5 - suggest updating via Apple Apps Store to version 11.5.2. Only once that is achieved, run the Software Update in System Preferences and update to version 11.6.

For the record, there are 2 reasons for the "Your system has run out of application memory" dialog box.

A) Your boot disk has very low free storage, and macOS cannot create page/swap files to offload virtual memory contents to disk. This is generally not the case, but I mention because if you do have very low free storage, it might apply.

B) A process (or set of processes) have asked macOS for excessive amounts of virtual memory address space. Virtual memory address space requires macOS to create Virtual Memory Page Tables in the kernel address space to keep track of the application virtual addresses given out. Generally, if there is a memory leak (process asks for a virtual address range, uses the addresses, forgets to give them back, asks for another virtual address range, uses the addresses, wash, rinse, repeat), eventually there are so many virtual memory page table entries trying to keep track of the virtual addresses, that macOS no longer has memory available for applications, and you get the "Your system has run out of application memory"

If you look at Applications -> Utilties -> Activity Monitor -> View (menu) -> All Processes -> Memory (tab), you can see what processes are using lots of memory.

You can also use

ps ax -o vsz,pid,comm | sort -n

which will display the virtual memory used sorted so the largest virtual memory consuming users are at the end.

NOTE: I have seen situations where no single process is using all the memory, but rather a host of smaller processes is using up all the virtual memory. That is to say, a 100 instances of the same program all running with a moderate amount of memory eventually adds up to a lot of page table entries being used.