Actually, it's not as easy as simply dragging it to the Trash. You need to Get Info (File --> Get Info) on the apps icon ( in the Apps folder, not on the Dock). Then, you need to add yourself under Sharing & Permissions. Then change your privileges to Read & Write. Finally, drag the app to the Trash.
Otherwise, just use your own app and ignore iTunes. No need to delete it really. Open With can be used to change all music files associations to whatever music app you prefer.