While I have no idea what you are talking about with your iPad being hacked and a backdoor was set up, but you can stop syncing your Google account with Contacts by going to Settings and choose Mail. Select the Google account used and one of the sync options will be for Contacts. You can also access this same information by viewing Contacts in your Settings and choosing the Google Account.
Also check what other apps have access to your Contacts by going to Settings > Privacy & Security > Contacts. Every app listed here has previously asked you for access to your Contacts and it was granted, but you can also revoke access to your Contacts. Any app that previously was granted access will still have that information, it will just no longer be able to access it or make changes. For example, Facebook will ask for access to your Contacts and once granted, then those contacts are stored in Facebook servers and they have that data. Shutting off access does not remove the data from the Facebook servers, it just prevents them from getting any future information.