iWatch SE on iPhone XR, swipe up/down randomly stops working

Hello there, less than a month ago I bought a new iWatch SE but I am randomly experiencing a bug for which the watch screen won’t open Notifications or Control Centre by swiping up or down. If I press the crown, I can navigate the apps by doing the same gesture, it just does not work -randomly- from main screen. If I reboot the device it will work -for a while- but in the last week I had to do it twice. I find it odd being a brand new Apple iWatch.

Any advice?