I actually just got going on this idea myself. I have about 300 GB in music, and 900 GB in photos. I bought a MacBook Pro Retina so I could move around and still be able to do all that I could with a desktop computer—now I can edit photos when I'm on the couch or the toilet! However, I couldn't have all my photos and music on my rMBP, given the huge space requirements of 1.2 TB, and it would be silly for me to spend all the extra money on a larger SSD when I don't need the space for more than multimedia, and 256 GB is enough for my OS, applications, documents, etc., everything except photos, music, and movies. I opted for putting my iTunes library on an external drive hooked up to my Airport Extreme. I copied it there and run it off the external drive. (Hold "option" when opening iTunes, and point to your iTunes library on the external location). Verify that it's all good in iTunes' Settings: (under "Advanced": look to see the location of iTunes Music folder, point it to the right spot if it's not, test out a song, and smile). A great aspect of this is that I can close down iTunes, take the hard drive with me and plug it in to my USB port, then reopen iTunes, and it will all be good to go, because OS X displays volumes in the same way, regardless of if they're connected locally or over a network. A downside is that I need to mount the drive from my Airport Extreme when I boot up, but OS X makes automatic mounting of network drives ridiculously easy: all you have to do is go System Preferences>Users>Login Items, click "+" then navigate to your network drive—no AppleScripts or command lines needed.
My hard drive is a 2TB 2.5" (probably 5400rpm) hard drive with a USB 3.0 connection—though I wouldn't expect the Airport Extreme would do anything faster than USB 2.0 speeds, but USB 3.0 is sure nice when plugged into the rMBP when I need to do it that way. The rMBP copies (compressed) files to the drive wirelessly at a speed of about 9.8 MB/sec, and copies compressed files from the drive at a speed of about 10.2 MB/sec. Although I don't have movies in iTunes, I have streamed 1080p movies over it, and it worked fine in VLC, without any of that pixely lag, but fastforwarding takes about 6 seconds to catch up if it hasn't buffered that far ahead yet. But you asked about music…
Except when starting up from having the Airport Extreme-mounted HDD spin down—which I suppose could be disabled—the access time is no slower than when I had an internal library. I'm not just saying that either 🙂 I have lots of music, mostly Apple Lossless audio. The important thing for me is skipping around, rewinding, or fastforwarding, and there is no slowdown that I can see when doing that.
Of course, always back up your stuff in at least one other place.
Hopefully this helps.