The time varies depending on your financial institution. Holds that I've had that were placed by other companies were typically removed in less than a week.
Removing your card from your payment method won't affect (won't speed up, won't slow down, won't stop) removal of the hold.
Also see the following from About payment card authorization holds from the App Store or iTunes Store - Apple Support
Authorization holds are removed by your card issuer after a short time. The exact time it takes to remove authorization requests varies by financial institution. If you use a debit card, you might see authorization holds as charges on your online statement. These charges are automatically reversed by your financial institution.
If you have questions about the way your card handles authorization requests, contact your financial institution.