Did this start happening after an update? Sometimes it takes a few days after an update for things to settle down. You may find these tips helpful:
To understand Apple Watch performance and its relation to your battery, see:
Apple Watch - Battery - Apple
Apple Watch Battery and Performance - Apple Support
“Battery life” is the amount of time your device runs before it needs to be recharged. “Battery lifespan” is the amount of time your battery lasts until it needs to be replaced. Maximize both and you’ll get the most out of your Apple devices, no matter which ones you own. See the "Tips for Apple Watch" section in:
Batteries - Maximizing Performance - Apple
If the battery drains too fast after an update, see all the replies to:
Update 6.2.1 Battery Draining Too Quickly… - Apple Community