Removal of calendar sharing

Hi all, hopefully you can help.

I am trying to stop sharing a calendar I own with one of the participants, however whenever I try to remove him, it says it has been successful and then he reappears in the list! I can't even amend his privileges to view only.

ANy thoughts? Many thanks!