How to restrict External Drive access in other user accounts

I just purchased an external HD and moved my iTunes and iPhoto libraries to it. No problem there.

My Mac mini has four user accounts and I'd like to set the permissions on the external HD so that only I can write to it from my account. I'm attempting to do this through 'Get Info' on the external drive, then setting Sharing & Permissions to 'Read and Write' for myself and 'Read only' for the other three accounts. But it seems that when I set one of these it gets set for all four accounts.

Am I doing something wrong trying to set the permissions this way, or, is there a better way to do it?

Thanks,
Andy