I have had the same issue and, like most who have responded, received the same run-around from Apple: "background app", "software issue", etc. I brought it into the Apple store about a week before Series 5 was released. They did the usual diagnostics and said battery health was good. When I pressed them to look further, they ran another diagnostics that showed battery cycles over several days and agreed that there seemed to be a consistent drain rate that resulted in about 8 hours of usage before it died - barely long enough to last through work day and make it home. They reset watch as a "new device" to avoid restoring any corrupted software (the standard line, it seems) and, surprisingly, scheduled a follow-up phone call 2 days later to give me a chance to monitor performance and provide an update to the support tech that would call. Sure enough, no change in the dismal performance and when the call came that Thursday, I was ready to "update" the tech. I lucked out with a great tech who seemed to understand the history, etc. and explained that because Watch OS 6 and iOS 13 was just released, I needed to update both phone and watch before they could continue since Apple doesn't support diagnosing issues on "obsolete" software. I did as asked, with no improvement, and spoke again to same tech the following Saturday at which time he said that if I would bear with them, they would like to run diagnostics on my watch over several days so that the software team could analyse what is taking place and hopefully result in a patch. I went along with everything and after sending the diagnostic files back to Apple, their engineering team determined that, surprise surprise, it wasn't software after all - it was hardware and that the watch needs to be repaired or replaced. Fortunately, I pursued this while under the original 1 year warranty and I am hoping to receive a repaired/replaced watch that actually performs closer to the 18 hour battery life we all were promised.
As a side note, it is interesting that when the Series 5 was released, the Series 4 was discontinued even though the Series 3 is still being sold. They say it is because of their multi-tier approach and the 5 is the newer "high tier" option and therefore the 4 is not necessary. However, I suspect that the hardware inside the 4 has been problematic since the beginning and they are trying to put it behind them is quickly as possible.
My advice to everyone out there - don't buy into the "software issue" theory. Keep pursuing this with your respective tech support. Go along with the methodic steps that they are suggesting and even offer to "aid engineering" by allowing diagnostics to be run, etc. Be patient as the process requires all the "t's crossed and i's dotted". Eventually, they should arrive at (read "admit") the same root cause that it is a hardware issue, not software and offer to repair or replace your defective Series 4 watch. Good Luck!!