Apple Account not working for wallet

I am still trying to gather information on this topic. Even if you add your Apple account to Wallet app it should only work with the App Store and Music. (There may be some exceptions)

What I have found is that it may be referring to an actual debit or credit card you have linked to Apple ID but not Apple Pay.

Adding credit or debit cards from an iTunes Store account Adding credit or debit cards to Apple Pay - Apple Support

For a credit or debit card on file with iTunes, the user may be required to reenter their Apple ID password. The card number is retrieved from iTunes, and the Check Card process is initiated. If the card is eligible for Apple Pay, the device downloads and displays terms and conditions, then send along the term’s ID and the card security code to the Link and Provision process. Additional verification may occur for iTunes account cards on file.

What I would do if I were you is contact the card issuer and see if there are any restrictions on the card you are trying to add and have them reset the tokens for the card. If you are solely referring to the Apple ID balance you cannot add that as far as I know.