I think I figured it out. Like all databases, usually items can't be deleted if there are trails of the thing you want to delete in the system somewhere.
Long story short, here's what I did and it fixed the problem.
1) Create a new contact called "DeletedContact", and put the number that keeps showing up as someone elses name. This renames the contact so you can identify it later.
2) Go through all your messages, and wherever you see "DeletedContact", delete (swipe the conversation to the right and click delete) the conversation saved in the Messages app. Note, this could take a long time because you'll have to go through every text conversation. You can always quick search how many you have to delete.
3) After you've deleted every conversation with the "DeletedContact" in it, go to the "Contacts" app, and delete the "DeletedContact".
This did it for me! Hope it works for everyone else!