I'm also experiencing Apple Music amnesia. I believe, just as Simon does, only I have much greater detail and I have both update my OS several times, I've completely reinstalled and I've upgraded my iPhone. None of which have resolved the issue. I've noticed that this is related only to tracks on Apple Music/Apple Radio.
Here's the scenario: lets say I open Apple Music and play a radio station. Then I pause the station and leave the app. If I don't interact with Apple Music for approximately an hour or so and then come back, it will have forgotten where I left off. I also want to note that it will default to the last thing that I played in my iTunes/Match Library. So Music does indeed remember the last song played. 100% of the time in fact. Only it will only remember what had been played from the iTunes/Match library and not what had been playing from Apple Music. This is the consistent behavior of the App for me and has been since the inception of Apple Music. No amount of updating upgrading or reinstalling has resolved this issue. It makes me believe that it is the intended behavior of the App.
Realistically, this isn't a huge issue at first glance as it takes little effort to fire up a Radio station again. Except... it completely breaks iOS's prediction ability. You know the feature where iOS tries to predict what you want to do next based on connected devices or environmental cues? For instance if I pug in headphones I get a now playing pane on the lock screen. This feature doesn't work after Apple Music forgets the last track that I've played. It does this consistently. I've been working with this issue now since Apple Music was conceived and released to the public. I've tried everything I can think of to fix it hoping that one day Apple would resolve it in a update. I've even gone so far as to upgrade all of my devices related to the issue including my apple watch and my Bluetooth adapter in my vehicle. I'm now working with an Apple Watch Series 3, iPhone X and a newly replaced Bluetooth adapter for my car. Apple Music still will not remember the last Apple Music track played and worse yet unless I was playing music on my iPhone within the last 30 minutes or so I will not get prompted to play music on my lock screen when I connect to my Bluetooth adapter in my vehicle. I know now that it this is related to Apple Music not remembering the last song played because the two consistently go hand and hand. If Apple Music still has the last song played qued and ready to go, I will get the now playing prompt on the lock screen. If it has forgotten it, I will not get the prompt.
I sense that this is intended behavior otherwise it would have been resolved long ago. But I can't get an answer from anyone. I've posted on reddit about this behavior, I've submitted feedback to Apple countless times, I've scoured the web high and low and no one seems to know why or how to fix the issue. Or if it is even something that needs to be fixed. I get the feeling that most people don't even know what the intended behavior is supposed to be because of how erratic it is, so no one really gets that it's an issue. Or if they do report it like Simon here, it is effecting them in different ways so it doesn't appear to be the same issue. The issue is that Apple Music prioritizes the iTunes Music library over the Apple Music library and defaults to the last played song in the iTunes library rather than treating them like equals and always remembering just the song that was played last regardless of the library. In return, this breaks the predictive functionality of iOS and the overall experience.