Disable automatic emails from Siri/iCal scheduling

I want to tell Siri to "schedule lunch with Joe at noon" so it ends up on my calendar, but so it doesn't email Joe as an invitee. Is this possible?

It'd be nice to have people associated with events on my calendar without spamming those people.