Why do "All Day" events duplicate in my Calendar app when I travel outside of my time zone?

I have Time Zone Support turned on so timed events show correctly in the various time zones as I travel. But events that are "all day" get duplicated and show as 2-day events - it's as though I've moved one time zone to the west, and the event that was supposed to occur between midnight and midnight, covering just one day, now cover 1 AM to 1 AM - across two days!

How do I delete the newly duplicated events that now clutter my calendar? Please don't tell me I have to turn off time zone support - that would not make any sense because that would make it harder to work across many offices and time zones.