2011 Mac Mini dual monitor setup not working correctly

Hi all, thanks in advance for the assistance.

I have a 2011 Mac Mini that I'm hooked up with both HDMI to DVI and Mini DisplayPort/ThunderBolt to DVI adapters. They never seem to work correctly. When both are connected and booting up, only the HDMI to DVI monitor shows up while the other one stays dark. If I unplug the HDMI adapter, then the Mini DisplayPort side of the monitor shows up. Plug HDMI back in and Mini DisplayPort goes dark again. Detect Display doesn't bring it back either.

Now, if I boot the computer with only the Mini DisplayPort connected, then it boots with that monitor showing, of course. If I then connect the HDMI adapter after it completes boot, them I get both monitors working. This basically is my process, but a huge PiA.

Anyone have any idea how this may be solved? Searched but similar threads I've read all ends up with the issues not resolved.