The issue is possibly related to the age of your Apple ID and the newness of your account and no real history in regard to texting, email, even location. Apple shares anonymous information about your usage and basically there isn’t any. Banks then make evaluations as to your fraud risk. This information is all automated and uses algorithms to detect fraud and other potential issues.
Visa would really only be involved if you had some fraud alerts. My suggestion is to add a payment method to your Apple ID. Subscribe to an Apple service, or subscribe to something and buy an app or two or even a movie or some music. Show some usage on your accounts so that it can be reported.
iPhones and Apple Pay are device specific. When you added the card to another device, it was sharing information about the Apple ID that was logged into and accounts associated with that Apple ID. So, kind of a double whammy, new account, and new device. Happy to try and answer any questions I can.