I ran into this recently, and in my personal case it was because my card was expired in my Apple ID vs in Apple Pay.
What I hadn't realized until recently is that my Apple payment in my Wallet and in my Apple ID were two different payment sources. Although my wallet was fine, the reference in my Apple ID was expiring. So for a while I kept trying to remove/re-add the card in my wallet (Apple Pay) when it was actually expired in my Apple ID area instead. In my Apple ID area, I was able to edit the expiration date and security code, and things worked well again.
Apple Pay: Open Wallet app
AppleID: Settings -> iCloud area (tap on your name) -> Payment & Shipping
That was my variant of the issue, hopefully it helps someone out there.
Note - I ran into this issue a few months ago as well, and I wish I had written the solution because it appeared differently. So it looks like different solutions might work for different people because I don't remember doing the above a few months ago, but it worked for me this time around.