Apple Watch elevation wrong - too high

I have the Apple Watch series 4 by the end of the day all my rings are closed which is great except I didn’t actually do any exercise. My watch continuously increases elevation, so I’ll have 170 something flights of stairs by the end of the day even if I didn’t do any. This is incredibly frustrating because I don’t know how much activity I actually did everyday.

I have it paired with an iPhone X. I’ve tried restarting both the watch and the phone, each are up to date, and all of the health and fitness are enabled under privacy. I’m stuck. Any ideas?