Turning off the function of long pressing the Apple Watch crown to activate Siri while working out

It just so happens that when I work out the back of my hand constantly activates Siri by unintentionally long-pressing the Crown on my Apple Watch. on my day-to-day, I like this functionality, but when I work out I have no use for it. and instead of being helpful, it is extremely annoying as it always interferes with my music.

Is there a way to turn this functionality off while on a focus mode or something like that?