Apple Watch heart rate inaccurate

So I’ve recently gotten an Apple Watch series 1 and I’ve been going on walks and so forth. Most of the time my Apple Watch will show my heart rate at 101 for example. Not two seconds later it will be at 82 or something. This is completely inaccurate and messing up my calorie loss. If I wasn’t over the date of return, I probably would have already returned it. If you have any suggestions, I’d very much appreciate it. I’d rather not have to take a loss on this watch since it’s expensive and find something else.