Resolution
3840 x 2560
ports:
USB-C x 1 (only for 135 W power supply)
USB-C x 1 (supports display, data transfer, and 65W max. charging power*)
USB-A 3.0 x 2
HDMI 2.0** x 1
MiniDP x 1 (DP1.2)
The speed at which screen data must be provided is far too fast to accomplish with the processor handling each byte of data. So purpose-built systems like the M1 Max use a Hardware display-generator automaton to send the entire screenful of data automatically, and the processor intervenes only once every screenful to set it up again for the next pass. The processor load generated is tiny, and could not possibly be what is causing your Gross symptoms of poor performance.
There are some ongoing issues with displays holding their resolution, but that is not your complaint.
There is a recently-discovered diabolical issue where, after sleep, the processor(s) may sometimes not fully awaken, and performance was cut drastically.
As a check for that 'failure to fully wake' issue, Restart your Mac, then without allowing it to sleep, run your test again.
Also,be sure you have applied all MacOS software updates, to avoid debugging issues that have already been seen and solved.