How do I tell what Ive downloaded music to my iPhone on Apple Music

I'm not aware of any way to see which albums have not been downloaded to your iPhone, you can only see which ones you have downloaded by selecting the Show Music Available Offline toggle in the My Music tab. Tapping on an album that has been downloaded will show a small phone icon in the upper right corner of the album art, but it's not easy to see on a small screen and doesn't appear unless you are already viewing the album in question.

If you don't use cellular to stream when out and about, your only option is to go to each album you want to have available offline and choose to download it. 😟 This isn't uncommon with streaming services, I've used Google Play and Rdio and it's basically the same deal with them as well.

It makes sense but it's pretty time consuming if you have a lot of albums in My Music. I usually just select certain albums to download, or I choose to download them at the same time that I add them to My Music, but then I also have enough data that I can stream without worrying too much so I don't have to download everything. Going back afterwards is a bit of a pain because there isn't any easy way to see at a glance which albums are downloaded and which are not. Hopefully Apple will change this in the future.