more than just the face changes

I am aware of the face change on the Apple Watch and can deal with it, but I have another problem that is harder to fix.

I use the Infograph Modular faceon my Apple Watch 4 and have the temperature reading (weather) in the bottom right of the watch face. For some reason it frequently changes to the UV reading on its own and the only way i can set it back is to go to my iPhone and touch the weather setting and set as current. The watch setting on my phone correctly shows weather in the bottom right so my phone didn't change it but I still have to set as current for the temperature to reappear on my watch. It is a quick fix but I don't always have my phone with me and I can't fix it from the watch.