With Wrist Detection and Passcode enabled, check that you are unlocking your watch after placing it on your wrist and that you are wearing your watch sufficiently snugly during your workouts.
Tracking can be affected if your watch loses contact with your skin, causing it to believe that it has been removed from your wrist. A loose fitting band can trigger this by allowing your watch to lift from your wrist and/or by allowing clothing (for example, the end of a sleeve) to slip beneath the back of the watch.
Better contact with your wrist may also improve heart rate sensor performance during your workout.
Apple suggests that you consider tightening the watch band before workouts and loosening it again afterwards.
Sensor performance can also be affected by temporary or permanent changes to your skin, such as some wrist tattoos.
Workouts may also be paused or ended early due to accidental interactions with your watch's screen (including by clothing). To help prevent this, you can lock the screen during workouts (see the Workout app article below for instructions).
If you are concerned about a possible hardware defect with your Apple Watch, then I suggest that you contact Apple Support (remote diagnostics and/or mail-in service may be available, if required), make a Genius Bar reservation or visit an Apple Authorised Service Provider for assistance: