Phone REFUSING to delete a contact photo

I am coming from an Android phone, and have all of my contacts on my iPhone synced up through my Google account.

For whatever reason, my phone will not let me delete the contact photo of one of the contacts. Whenever I delete the contact photo, it seems to reappear within seconds. At one point, I deleted the contact photo, but then it picked up another contact photo from one of the linked emails on the contact, and then would not let me delete that contact photo.

Just wondering if anyone else has noticed this and has found a workaround?

The contact has a couple of gmails linked to it, and it seems like maybe it is picking it up from those accounts.