I posted the question on Reddit and got the following detailed response. For me re adding the card (technically for a second time) seems to have at least temporarily solved the solution. The response I received elsewhere is copied below.
Cards powered by some payment systems (e.g. mastercard), provide language preference list to the terminal during communication flow. If received, the terminal changes the display language, this was intended to improve international shopping experience.
The value of this list is set during the card provisioning process, the initial selection of available languages is provided by the bank, but the priority might be changed depending on the language and region priority set on device during the set-up process.
“My advice for you would be to check the languages list on your phone, then try removing and adding a card again. If this doesn’t help, then it is probably on your bank and their configuration.
Another, albeit a less possible reason, is that your apple pay card doesn’t provide the language list, therefore terminal chooses to use the local language instead.”