Always on altimeter values changing slowly in the same location

Hi,

The always on altimeter on my Apple Watch SE is behaving strangely. It will change values slowly (usually from a lower value upwards) while remaining in the same location. For example, sometimes after a restart it will return to the known correct value of 140m and then over a period of a few hours, rise to 260m (without actually moving). I have tried to recalibrate by going for a 20 min walk and tracking it in the fitness app. Over the course of the walk the altimeter adjusted from 260m down to 210m at the lowest point. It should have gone from 140-90m (I know this from an elevation website that uses gps). This is the same range of 50m so it seems to be starting off on the wrong number, but is adjusting accurately relative to the incorrect starting number. Any ideas on what I can do?

Thanks in advance