Apple Watch complications blanking out when screen dims

On some of the watch faces, Apple Watch blanks/whites out complications when the screen dims. It often doesn't come back when I turn the watch face upward so I have to turn my wrist a couple times to get it to show up. Or sometimes there's a very long delay. The attached photo shows an example of how the calendar at the top just displays "CALEN..." instead of the date, and the step counter numbers at the bottom is whited out.

Is there a setting to fix this? It's super annoying.