I’m trying to change my payment method but it won’t let me add a new card or delete the old one.

Try to verify your payment information, you need to Sign in to your Apple ID account page and edit / update your Apple ID details -> https://appleid.apple.com                                                     

• On the Sidebar click Payment methods -> then tap Payments Methods and tap Manage payment method.

or

If you need to change or update your Apple ID payment method

If you need to change or update your Apple ID payment method - Apple Support

• If you want to change your card, you need to add first the new card, then remove the old card.

or

Add a payment method to your Apple ID

Add a payment method to your Apple ID – Apple Support (UK)

• If you want to change your card, you need to add first the new card, then remove the old card.

See also the following Apple articles:

Payment methods you can use with your Apple ID

Payment methods you can use with your Apple ID – Apple Support (UK)

fyi: Not all credit & debit cards seem to work with App Store / iTunes Store, try a different one, or preferably a credit card.

If your card is not accepted as a paymenf method, you need to contact your bank.

Remove a payment method from your Apple ID -> Remove a payment method from your Apple ID - Apple Support

⤵️

If you can't remove a payment method

If you have subscriptions like iCloud+, have an unpaid balance, or you use purchase sharing, you must keep at least one payment method on file.

• Do you have a subscription? You can cancel iCloud+ and cancel other subscriptions from Apple. When your subscription period is over, remove your payment method.
• If you're in a Family Sharing group and you use purchase sharing, the family organizer must have a payment method on file. If you're the family organizer, turn off purchase sharing, then remove the payment method.
• If you have an unpaid balance, you might not be able to remove a payment method. Instead, change your payment method.
• If you still can't remove a payment method, contact Apple Support.

If you have followed all the steps and instructions and if you still need help, get support from Apple directly -> Get Support or call Apple in your region see -> Contact Apple for support and service or check -> here for more support and assistance.