Hello,
Thank you for posting to the Community. I'd be glad to help you with your concern regarding Auto-complete suggestions.
Once you have deleted a contact, you will not be able to search them in your contacts list on People page. However, in your Inbox's address bar (To: field), Outlook.com will suggest people (their email addresses) you've communicated with, even mistyped email address/es. To remove the specific contact in the auto-suggestion list, hover your mouse pointer to the X mark beside the address bar. You can also set your account to only suggest people in your contact list by following the steps below.
1. Log in to your account.
2. Once your Inbox contents are shown, click the gear icon right beside your account name found on the upper right hand corner
3. Select Options in the drop-down menu
4. Under "Customizing Outlook.com", select Advanced privacy settings
5. Tick the radio button beside Only suggest people in my contact list in the Auto-complete suggestions
6. Hit on Save for the changes to apply
If the steps helped you, please come back and let us know, if not, we will be happy to help you more.
Office 365 and Exchange accounts have a "Suggested Contacts" feature that has caused this problem for me. When I send messages through my Exchange account to email addresses not in my Exchange contact list (even if they are in a contacts list under another account), Exchange automatically adds their email address to my Exchange contacts. While there is a setting to disable this if I was using the Windows version of Outlook 2010/2013, I'm using Apple Mail, which means I cannot correct the problem. There doesn't appear to be a setting through my (833) 506-1666 Office 365 online account that controls this "feature", either.
To address the issue for now, 1-833-506-1666 I have to manually clean up the Exchange contact list periodically. Thus in my case, this is a Microsoft problem, rather than an Apple problem. That may be why so many answers to this question are, "Mavericks/Yosemite does not do this".