Screen sharing problem

I had no problem accessing my Mac Mini with Lion (both connected to a giga switch which is then connected to my Mac Pro ethernet plug). Ever since upgrading to Mountain Lion I get this message.

Connection failed

Please make sure that Screen Sharing (in the Sharing section of System Preferences) is enabled on the computer to which you are attempting to connect. Also make sure your network connection is working properly.

The fix I have is to unplug the power of my giga switch then I can connect again and screen sharing works.

Any idea what's going on here and how I could fix this?