How do I stop 'Maybe' notifications for a split contact in Messages on iPhone?

I recently split a single contact, that had an associated personal and business number, into two separate contacts because Messages kept choosing a number (either the personal or business) at random when I tried to text my contact. Now when I receive a message from this contact, it says "Maybe (known contact name) and (business contact)." I find the "maybe" message annoying, but don't want to delete the conversation and start fresh to make the notification show only my contact's name since the existing conversation has an even mix of the business number and personal number due to Messages' fuzzy logic.

I would be fine re-merging the personal and business contact into one if I could specify the recipient phone number instead of Messages randomly picking between them. If I can't, I would like to have the conversation associated solely with the personal contact and have it not tie the business contact into the conversation in notifications.

[Re-Titled By Moderator]