You can generally bank on your Mac actually supporting the specified number of displays, provided you are really following all the subtleties in the specifications, and using high-enough spec cables that are appropriately short.
The data rates needed to re-paint almost any display data are FAR faster than can be accomplished by ordinary one-character-at-a-time I/O using the processor. They can ONLY be accomplished with a hardware assist, which I refer to as a Hardware display-generator.
The fixed number of these devices included in your processor is generally the only limiting factor. Once the screen image has been rendered into the display buffer, the graphics processor(s) are not involved in any way, until you change something.
The display-generator device automatically fetches successive characters from the display buffer in memory, serializes, and sends them out to the display. Only when the entire screen data have been sent does it make any demands on the processor. it interrupts once, asking to be set up again for the next entire screenful.
Supporting additional displays when you have an available display-generator adds only a trivial amount of processing. What it needs is memory bandwidth. ALL the Apple silicon M-series processor are memory-bandwidth BEASTS.
Executive Summary: Add as many displays as the specs allow, and there will be NO measurable impact on the overall speed or performance of the computer.
