Both my wife's Apple Watch Series 4 (Watch OS 7.3, iPhone 7 iOS 14.4)), and my Series 6 (Watch OS 7.3, iPhone 11 iOS 14.4) are having this same issue. We noticed this when her Map My Walk app didn't seem to be updating her activity, and I noticed that, even though the app icon appears on her watch AND opens/works, the icon in her Watch App on her phone show it is not installed.
So, I restarted both her watch and her phone, then I uninstalled the App by using her Watch face, then tried to install it from the Watch App on her phone -- no dice. The install icon spins for a few seconds, then goes right back to saying INSTALL -- and the app does not appear on her watch.
So, I went to my Series 6 and randomly tried to install an app that isn't installed on my Watch -- I tried to install my bank app, the Duo app, the Apple Store app, and the Bible app. None of them will install.