Please send feedback with a feature request directly to Apple using the feedback form: http://www.apple.com/feedback/photos-ios.html
You are right, we need a way to remove the music from the memories completely. It can be very embarrassing to have the music start automatically, whenever we accidentally start a memory to play. And I cannot even select most songs from my Music Library. None of the soundtracks I created with GarageBand are eligible, only Apple Music. The songs I created myself are all giving error messages, when I try to use them.
I am making sure, the sound is turned off completely on my Mac or iPhone, before I start to work with Photos.