How do I delete an empty VIP Mailbox? - there are no more messages from the sender in my inbox

I would like to delete an empty VIP Mailbox for a sender that no longer sends messages to me - there are no messages in my inbox to remove the star. I tried going to Finder/library/mail - but VIP files reappear after I try to delete them.