I solved this problem doing the following (sorry if this has already been posted):
1, on my iPhone I turned off iCloud Music Library
2, connected my iPhone to my Mac and through iTunes and removed all music
At this point my iPhone had NO music on it.
3, I then added back the songs from iTunes, copying them via USB
4, Once everything was copied back I turned back iCloud Music Library. This time I was prompted with a message asking whether I'd like to "Merge" or "Replace" the existing songs on my iPhone. I selected "Merge". It took a few minutes to update my phone's music library...
And voila, I had a single copy of every song on my iPhone. Songs I had copied via USB have the little grey icon next to them. Songs that can be streamed via iCloud do not.
Here's the point: I think having existing music and then adding your iCloud library creates a conflict and duplicates songs. You have songs preloaded, and iTunes uploads your complete library to the cloud and then downloads them to your phone, not merging the cloud songs with the existing ones.
So I think that you could probably solve this issues by doing steps 1 and 4 and selecting "merge".
Make sure you're iOS is updated on your iPhone because I have a feeling that the "merge or replace" prompt is a new addition.