Had the same issue on a new Mac Studio, brought this month, running Monterey.
Apple support will tell you to boot in safe mode, then reinstall macos before your support request can be escalculated, which is a waste of time.
THE FIX - Preferences > Energy Saver > Check 'Prevent your mac from automatically sleeping when the display is off'
It appears that when these m1 macs sleep, then wake - the connection to external monitors (I have 2 dell monitors and a Wacom Cintiq) doesn't register instantly, so our app windows are getting moved to the main display, then scattered across connected displays once they are recognised.
The above fix worked for me but this is clearly a macos bug that needs to be sorted by Apple