Apple Watch workout app keeps pausing during cycling and skiing

My watch workout keeps pausing while I'm downhill skiing (mostly on the chairlift) and when I'm cycling (mostly when I'm mountain biking when I stop to chat with friends, but sometimes also on my road bike). It pauses and does not automatically re-start when I get going again. It also seems to stop if I don't have a sleeve or a glove covering it. Any explanations and advice? Thanks.