How to remove calendar default invitees PERMANENTLY, not each time you create an event

When I create an event with a name that has occurred before, like "Dentist" my ex pops up as an invitee. How do I remove my ex as an invitee permanently from calendar, regardless of the event created? I have already deleted him as a contact, but still shows up. I don't want to have to keep manually deleting him each time I create a calendar event. Please help me delete him from life, PERMANENTLY! FYI: this is not a shared calendar but a personal one, where I had invited him to specific events as they were created.