Apple Watch Ultra 2 - ‘Allow Compass to use your location’ doesn't work

I recently just got my AW U2.

Version: WatchOS 10.1.1

I use Compass for a complication. Whenever I click Compass app/complication, Watch asks me to ‘Allow Compass to use your location’. I have enabled this ‘While using App’ and ‘Precise Location’ multiple times but it still pops up (at least 8-10 times just today). Checked my iPhone and it’s there also. Subsequently, my iPhone requires me to ‘allow location’ multiple times too when I open the Compass app.

What I tried:

1) Deleting compass app on iPhone and restarted both devices. Downloaded compass app again but still asked me to allow location.

2) Unpaired watch and reset. Still didn’t work.

Not sure if this is hardware or software issue. If it’s hardware, I can return it still.

Only on my second day of the purchase man...

[Edited by Moderator]