Are you using iCloud? Is someone else using the same account? If so, they may be affecting your contacts.
Try to have only one service synchronizing your contacts. If you're using more than one, for example iCloud, iTunes, MS Exchance, ... there my disappear some contracts.