YouTube sort of “resets” if I pause a video and web browser for more than 30 seconds

I’m at the latest os version. YouTube seems to reset and “lose” the video I was in whenever I pause the video for more than 30 seconds. Sometimes if I’ve put it in the background and sometimes even if I left it on the YouTube app. I have YouTube premium. The video is in my watch history and goes right back to where I was previously, but what a nuisance. Is there a fix for this?