Having the same issue with app on the iPhone. iOS up to date. I even paused the show "See" (didn't close the app), then went to another app for just a minute or less. Went back to go watch, and it exited the show. Then went to watch the same episode I started, and it started at the beginning. Also, I don't see why they added the "+ Add" button, if the Apple show doesn't even go to the Library tab. I think anything you add should go there, so that you can get to your shows quickly instead of having to find them when you open the app.
Watching in the browser is even worse. I'm on PC and tried in both Chrome and Firefox. Chrome rarely even starts the show, and if it does, it's literally like 5 minutes later. Firefox was a little better, but took forever to start as well. And they both have the issue of not knowing where you left off. But I bet if I was using Safari on a MacOS laptop/computer, it would probably work perfectly.
This was obviously a rush job to get Apple TV+ to market. I hope Apple can address the issues to turn this into an actual app worth paying for.