Changing the email address from which iCal responses are sent

I currently have four email accounts set up in Mail. When I receive iCal event invitations, my responses are being sent from the least appropriate of these four accounts and I was just wondering what I needed to do to make sure my responses are being sent, either from the account to which the invitation was sent in the first place, or at least from a more appropriate account than the one currently acting as default.

Here's what I've tried already:

- Changing the preferences in Mail: No luck, it doesn't seem to have a category for this action

- Changing preferences in iCal: as above, unable to change the "responding to invitations" preferences, nor the default account from which responses are sent

- Editing my contact card in Contacts. All the relevant accounts are in my card, but can't work out whether one is set as a default that interacts with iCal

I'm using a Macbook Air running OSX El Capitan (10.11.2)

Can anyone help out?