No, a restore from backup is all or nothing. However, if you use iCloud and sync contacts, they are not part of the backup anyway and will just sync back when you sign into iCloud. Same goes for photos if you use iCloud Photo Library or sync them from a computer via iTunes.
Only the photos in the camera roll folder are ever part of a backup.