Here's the response I got from Apple support:
'regarding the offline function you might see, you will only see that when browsing the Apple Music catalog with Apple Music or your past purchases. You have the option to do this for items on your device that are not currently available for offline mode. Otherwise it will normally stream the music. This is an optional function, and will only appear for the items that are not available for offline use at that time.
If you would like to see what you have available for offline use… Go the the “My Music” section of your app. Under the section where you normally choose sort by "album, artist, song name” and so on, if you tap that there will be an option to choose “Show Music Available Offline”. If you choose that, it will show you what is presently stored on your device. Here is an article that covers offline use in more detail: https://support.apple.com/en-us/HT204839
....which makes absolutely no sense to me, as I'm not browsing Apple Music - I'm browing My Libary on my device which is offline. Apple Music is turned off.