You cannot do this on an iPhone. Two indirect ways come to mind:
Restore the iPhone as new. Then add only items other than contacts (e.g., photos, music).
Consider where the contacts are housed if they are not local to the iPhone. For example, are the contacts in iCloud? Then delete them from the place that houses them.