Two computers, one monitor. Switching back to the Mac mini gets no video out

I have a two computer, one monitor setup at home, with a MacMini 2018 Intel, and a generic PC (for work...:-(.

When I switch Inputs from the Mac to the PC, everything works fine, but when I switch back to the Mac after being on the PC for a while, the video on the Mac does not come back. I know it's alive, because I can ssh into it, and when I remote control from my other Mac, I can see the screen and manipulate it.

When this happens, I have to either remote in and reboot, or at worst, hard shutdown with the power button.

How can I make sure the MacMini video "wakes up" after I switch the monitor away (HDMI, by the way), and then back?