AirPlay System Audio problems

Has anybody else had this problem?

I use AirPlay to stream audio to my AirPort Express quite a lot, and for situations outside of iTunes (Spotify, web audio, etc.) system AirPlay is really useful. Unfortunately, it's really unreliable and frequently messes up.

My biggest problem is that really randomly it refuses to change AirPlay devices. I hold down Alt, press the Volume menubar icon, select my AirPlay device and it just ignores it entirely.

If I go to System Preferences, Sound and select my AirPlay device, it just jumps back to Internal Speakers.

The only thing that fixes it is a restart. Which then leads me to the other problem, that the Alt menu only brings up one AirPlay device and I have like 5 different ones, so I have to go to System Preferences to see the full list. It remembers the last used which is good, but restarting clears that.

I'm using a Retina Macbook Pro with the latest Mountain Lion, all AirPort Extreme and Express base stations, just playing audio through the Express optical port.

I'm thinking it might have something to do with a conflict with iTunes built-in AirPlay, which continues to work even when I have this problem. It could be that using system AirPlay, then using iTunes AirPlay causes some problem and makes system AirPlay stop working. Quitting iTunes doesn't help though.

I should note that I can continue to use AirPlay from other devices to the AirPort Express or Apple TV, and as I said iTunes on Mac can continue to switch between AirPlay and internal fine, it's just limited to Mountain Lion system AirPlay which randomly decides not to work.

AirPlay video mirroring ***** as well but that's another story. Suggestions for fixing, or just crappy programming?