You have done the standard DIY checks and confirmed there is a problem, the next stage is help direct from Apple Support. Your profile suggests Series 4 so your watch is still in warranty. Contact Support (link) and get an official support case started. They might be able to do remote diagnostics over the network, or simply arrange a Genius Bar appointment (or mail-in service if it is more convenient for you) to get your watch and charging accessories checked.
Adding: Hint, try to get a Genius Bar appointment if you have a store within reach, because that means you can demonstrate the problem face to face and also get the watch and charger checked as a combined set.