Alerts not going away

I keep getting alerts popping up for some Exchange calendar events that I want to go away. I've gone into the event on iCal and set alerts to "none," but 5 minutes later another alert pops up and the event has its alerts set back to the way they were before. The alerts keep popping up every five minutes, even if I don't do anything! How do I get this to stop?