I am having the same issue with my Apple Watch Series 6, OS 9.1. It was worse until I renewed my Applecare+ since my initial warranty expired this month. It's still there but not as bad. I thought it was a hardware issue until I started reading this thread and saw many people having the same issue. Hopefully it is software and it will be fixed soon.
I wonder if it has something to do with the Applecare+ renewal message that is in the watch app on the iPhone.
To see your AppleCare+ status, open the Watch app on the iPhone, and navigate to General->About->AppleCare+.