Depending on where the groups and contacts are located. If all the contacts are in iCloud and the groups are actually under the local On MyMac contacts, when you drag a contact to the group, it is also copying to the local contacts since that's where the groups are located.
If you continue to use iCloud, go to system preferences, open the iCloud icon and sign out. Select to delete contacts from the computer.
Then sign back into iCloud and contacts and groups should merge. You should only have iCloud contacts now and On My Mac should not show.
Unless you are using local contacts to sync to something else, it's not needed.
You should also make sure you have a backup of your contacts such as a TM backup. If, archive your AB before doing any changes in case something goes wrong.