Siri won't play music stored on my phone

Whenever I ask Siri to play songs stored on my phone she won't always be able to claiming that I need to be on a wifi network or have cellular data turned on when I know for a fact i've stored these songs on my phone. I'm using Apple Music with my phone which I feel may be the issue as the songs play fine when I'm at home and on the wifi network. I've read on various forums to specify to play songs in My Music as if you only ask for any song from a band it will try to play anything from that band non Apple Music. I'm aware some may suggest just taking out my phone and selecting it manually, but I'm trying to use this feature whilst driving and using it manually is besides the point, it should work without such issues.