I need to remove a number from my Apple ID

I’m trying to remove a phone number for my Apple ID yet he keeps telling me to sign out of FaceTime and iMessenger I can’t seem to do that I just wanna remove a number I have two numbers on it I only want my number on it