If you have iTunes Match, yes the idea of that is that you have access to all your music from anywhere. If you access it though you will be downloading it, in order to play the music you have chosen to play, once downloaded, that music will stay on your device unless you delete it.
Music that is on your phone will show up as normal, music that is only in the cloud will have a cloud icon next to it, when downloaded the cloud icon will disappear, if you delete it again the cloud icon will disappear again.
In the example below, all my albums by Matt Schofield are on my phone, the others would need downloading.
Photo's are different they kept on your device and only synced to the cloud so they can be pushed to other devices.