You may be wearing you watch too loosely and inadvertently pressing the side buttons or swiping the watch face:
Wearing your Apple Watch - Apple Support
The first (and simplest) thing to try if something isn’t working right is to restart your Apple Watch and its paired iPhone. If necessary (device not responding), try a forced restart. Both methods can be found here:
How to restart your Apple Watch - Apple Support
Restart your iPhone - Apple Support
If that doesn't help, the next steps would be to unpair and pair your watch again and restore from the backup just created. This should result in no loss of data. It deletes all temp and corrupted files and re-indexes the file system.
Unpair and erase your Apple Watch - Apple Support
Set up your Apple Watch - Apple Support
It may be a hardware problem, but for two watches to do the same thing is rare. However, if it still under warranty (most Apple products come with a 90 day warranty and complementary Apple Support), I wouldn't fool around with it much more. The quickest way to get it resolved is to make an appointment at the Genius Bar in an Apple \Store near you or contact online Apple Support. Either way:
Genius Bar Reservation and Apple Support Options - Apple