It's not that you are doing something wrong, that is the way that syncing with iTunes works. iTunes will sync what is currently in its library on your computer. If it is not already in your iTunes library, then syncing will replace (and therefore erase) whatever is already on your iPad with whatever is in iTunes on your computer.
If you have never synced with iTunes before, you can make the iTunes library on your computer your syncing iTunes library. You do that by transferring purchases from your iPad into iTunes, backing up the iPad, restoring from the backup and then syncing with iTunes. This user tip explains how to do it.
If you don't want to go through all of that, even though it really is easy to do and well worth the effort - IMO - you can simply drag and drop the songs onto your iPad from iTunes if you set iTunes to manually manage music and videos. This explains how to do that.