Am I using up my data when I am listening to apple music and I am not on wifi? Also if I an listening to my music but have songs on my playlists that I got from apple music but have not made available offline am I using my data listening to them?

(1) Yes - if you have not downloaded (or in Apple talk "make available offline"), yes you are using either Wifi or Cellular data to stream the music.

(2) Yes - that is what "Make Available Offline" is for. You can make any song/album/playlist available for offline. The iPhone will download those tracks - preferably while you are on Wifi. You then go to My Music/Library screen. Tap on the view list (generally in red says Songs/Albums/etc.) and at the bottom toggle on "Only Offline Music". This will change the Library view to show only what you actually have on the device.

Be aware - you can still play selections from the For You, New, Radio and Connect screens, and playlists you added from Apple Music - and those will still be streamed.

Makes no sense to "turn off" Apple Music as you have to pay the subscription fee just to make use of iCloud Music Library to even download Apple Music tracks to your iPhone. Just don't select anything from For You, New, Radio and Connect if you don't wish to use your cellular data. Keep the Music app on My Music and Library selected. Then turning on "Only Offline Music" will show what is available offline. (BTW - there is still a small amount of data used even listening to offline music. My guess is this is what iOS devices need to keep confirming that you have a current Apple Music subscription. But the amount of data is minimal compared to what you would need to actually stream music.)