How do you turn off automatic smart stack

On the Apple Watch Ultra 2 (2023 version) since the last watch update to 11 when I raise my wrist to check the time or my heart rate its always stuck in the new 'smart stack' page. I can click the crown to exit, then re-navigate to either the watch face with my complications or restart the app it covered (usually the exercise app).

After some experimenting the issue seems to be if the crown rotates clockwise 1 click (say from a jacket, shirt, or resting you hand in your lap) this smart stack page comes up and never goes away.

Apple support was pretty unhelpful at first, their best suggestion was to submit feedback. In the middle of working on an apple care refund the next agent did come up with an OK workaround that helps (but doesn't fully fix it): on the watch itself in settings (this is not visible on the phone app) you can turn off 'wake on crown', for me this helped 80% of the time and I can now see the time on my watch like I could a couple of weeks ago.

If the watch is already awake it will still shove you into the smart screen, so it doesn't work while working out with the heart rate/timers/etc running on the screen (if my coat touches the crown while jogging I have to stop and renavigate back into the app to see my heart rate again).