I assume that you have set up "hotmail" mail account on your iPhone 4S and included the contacts from Hotmail when setting it up. Here is what you can do.
Go to "Settings" then "Mail, Contacts, Calenders" then select "Hotmail" from the list. Now you should see your hotmail account name on the top and 4 options on the bottom, namely Mail, Contacts, Calenders, Reminders. TURN OFF Contacts (Also Calenders and Reminders if you don't want them to be included). Leave only the "Mail" ON. Turn your phone OFF and ON from the top button. And this should solve the issue.
Include the details of your previous phone (Is it an iPhone also and if so, what i-OS is it running on). One option is to sync your previous phone's contacts with outlook express (depending on your make of the phone) and then you can sync all those contacts with your new phone. But include the details of your old phone and I would be in a better position to suggest different ways of doing it then. (Also include where are your new contacts now? In the PC (outlook) or in iCloud?)