The Thunderbolt/MiniDisplay Port on both the Server and Standard 2011 Mac Mini's support up to 2560 X 1600.
From what you have described it sounds more like you have a bad or shorting Port and I'd take it in to be checked out.
I'm having exactly the same problem. In my case, my normal config is with Mountain Lion and adding the WQHD monitor worked fine. However, when I rebooted into a Lion partition, it behaved as you described.
The problem is, I HAVE to run in Lion for work, since I'm dependent on a 32 bit application and Apple discontinued 32 bit support in Mountain Lion. So, I still need to pursue a solution for the Lion config. Anyone with insights?