I assume you have the "Notifications Indicator" turned off in your watch settings.
Try pulling down from the top of the watch to see your notifications and then force-pressing to clear all notifications. Once you do that, your yellow dot should stay on all of the time.
With the Notifications Indicator setting turned "on" and there are waiting notifications, the yellow dot shows for a second and then is replaced by the red dot as you'd expect. However, when the Indicator setting is "off" and there are notifications waiting, the yellow dot shows for a second and is replaced by a black dot instead. So, it seems that the "Notifications Indicator" setting doesn't REALLY turn off the notifications so much as make the red dot black.