Duplicate shared events in shared calendars

My partner and I each have our individual calendars that we resepctively share with eachother. When I invite my him to an event and he accepts it shows up twice. In his and in mine. Anyone know a way to not show an mutual event twice? If I unclick his shared calendar, yes it goes away, but then so do all the events I want to see that aren't ones I'm already attending...

I can't seem to find a solution. Thanks!