I recently added a visa gift card to my Apple payment account and not it’s not letting me download anything including free games, so I went to remove it both in the AppStore and online and it is not allowing me to do so.

What does 'not letting me download anything' mean, what happens ? If you are getting an error message what does it say ?

And for the removing of the card, how/when are you trying to do it ? You won't be able to remove it when trying to download something, you need to just log into the account: Change or remove your Apple ID payment information in the iTunes Store - Apple Support

If that is what you are doing and 'not allowing me to do so' means that you aren't getting the 'none' option : Why can’t I select None when I edit my Apple ID payment information? - Apple Support

If you log into the account and view the payment details screen are there any errors ? Have you tried removing the card exiting the payment details screen, and re-entering its details from scratch (or leaving it as 'none' if you only want to download free apps and updates) : Change or remove your Apple ID payment information in the iTunes Store - Apple Support ?