Remove a phone number from my apple ID

I recently switched to a new provider and kept the same phone number. However, in the “contactable at” section of my Apple ID it shows both the correct old number and the temporary number I had.

I want to remove the temporary number but I can’t find any way of doing this. I no longer have access to the temporary number.