Can't Swipe Up or Down on Apple Watch

From time to time I can't swipe up or down to reveal the Notifications or the Control Center on my Series 5 Apple Watch. My wife doesn't seem to have the same issue on her Series 6 but I see where other users have reported having the problem on 6 so don't think issue is related to hardware but more likely watchOS.

While the workaround is to restart the watch, the frequency of it becoming unresponsive to these swipe gestures, I'm currently having to restart over two days when the problem happens. No, I haven't wiped the watch completely clean and started over again, can't say I've ever had an issue resolved with that step but looking to see if others have this, other workarounds and will look to report this as a bug that hopefully can be addressed in an update to WatchOS.