While there is a bit of wiggle room for the metal ring to snap onto the rubberized magnet around the gold contacts, it is important that the connector body be able to connect perfectly flat with respect to the body of the Mac. That will allow the gold pins in the plug to make full contact with the gold pads in the Mac socket and allow the full 85W of power to be transferred without overheating the connection.
I have a Speck hard shell on my Mac, and I had to sand down a bit of extra plastic leftover from the molding process in order to allow the connector to lie flat when the cable is let out the back (the connector is "reversible": cable can go out the back or out the front, as needed, although out the front will interfere with the ports.
The new Retinas out this year come with a MagSafe 2 connector, that appears to be somewhat larger than the traditional.
More info (check out #7 & 8, click on triangle to open, with respect to bad contacts): http://support.apple.com/kb/TS1713