iPhone X screen wearing-out around the speaker

My guess is you are wearing off the Oleophobic coating on the iPhone. It's not really a screen issue, but one where perhaps your fingers are a bit more acidic than other users. You would be a prime candidate to use a screen protector and even they might wear a bit faster, but then you could replace often as they're pretty inexpensive.

It may help if you post an image of the screen wear that you are referring to. Also do you normally use a case or screen protector with your phone? If so, could it be possible that this may be the cause? Have you taken your phone to your local Apple Store to see if they concur with your assessment and offered you any options going forward?

Visit an Apple Store or Authorized Apple Service Center. Let them assess your iPhone and advise you how to proceed.

There has been a different iPhone X issue described here. This does not sound like yours.

iPhone X Display Module Replacement Program for Touch Issues - Apple Support

Other pertinent reading:

About the Super Retina display on your iPhone X, iPhone XS, and iPhone XS Max, and the Super Retina XDR display on your iPhone 11 Pro and iPhone 11 Pro Max - Apple Support