I've had 12.3.1.23 on here since October and I've been running El Capitan since it came out on my 2012 Mac Mini Quad-i7 Server (2TB RAID 0 with 8GB RAM, Memorex BD USB3 drive, dual monitors and two 7-port USB hubs connected that are about 2/3 in use, multiple printers, etc. etc.). Only today (trying out my new Logitech K740 illuminated keyboard's extra media keys) have I had problems. However, I don't always listen to music on my computer itself. I use it more as a server for the rest of the house and didn't have any trouble there with AppleTV and XBMC (now Kodi) playing my music library.
But I DO use it directly every day when I go to sleep as a noise maker (brown noise 1.5 hour file on repeat one sent via Airplay to my bedroom's Airport Express receiver with Klipsch sub/sat system). I have not had any issues playing that file, but that particular "song" runs off a RAM Disk (created via TmpDisk) so that I don't put excess wear on my 3TB media drive for something that runs 6-8 hours every day (I have to manually move the file over to the ram disk when I do a reboot, but fortunately I don't reboot very often, it being OS X and all). But while playing with the keyboard, I started having stuttering and massive beach balls errors (playing songs off the media drive instead). At first, I thought it was my hard drive going bad. I had to reboot using a recovery USB stick I made after putting in El Capitan due to my system running internally off a RAID 0 setup (no recovery partition when you go RAID). The Disk Utility said the drive was OK as was the internal RAID drives (had to boot Mavericks install/recovery to check the RAID drive integrity since El Capitan's drive utility is really awful). Playing the songs from Finder worked fine so it was obvious there was no corruption.
After reading this thread about the Store issues, I just logged out of the iTunes Store (the problem here does not start until I play something) and also turned off the "Connect" feature. iTunes 12.3.1.23 is now running 100% normal speed and not beach-balling or locking up at all. I've just played through about 6 songs now with no issues. So whatever is going on (here at least) is related to the iTunes Store connection. The thing is I've been using it for the past month with no issue (the store and syncing, etc.). It's only when playing music off my media external hard drive with the iTunes store logged in that there was an issue. Since I turned connect off at the same time, I can't be sure if that had any effect, but I kind of doubt it based on other posts.
I've also used a family member's 2012 13" Macbook Pro with an AppleTV setup and again no issues with AppleTV playing back the shared content what-so-ever. I didn't play media directly from iTunes on her computer due to the awful notebook speakers, but I did Airplay some things to the the same AppleTV directly from iTunes 12.3.1.23 and it didn't lock up.
Has anyone having this problem tried playing music over Airplay to other speakers instead of the computer's internal outputs? Given it didn't act up at her house using Airplay, I have to wonder if the problem may also only occur with internal playback. That might explain the brown noise here as well (since I airplay it to my bedroom, but I have used it locally in the room when the neighbor plays basketball sometimes to drown the ball sound out if I need to concentrate and again from the RAM disk and it didn't crap out so again it points to hard disk playback.
One thing I've noticed since moving from Mavericks to El Capitan is that concurrent disk activity on conventional (even RAID 0) hard drives appears slower now than before. I used to get 230MB/sec in Mavericks from the RAID 0 setup and now I get 160-185 at best in El Capitan so I don't think it's my imagination that Apple botched something somewhere between Mavericks and El Capitan in terms of conventional drive access (that Macbook uses a 5400 RPM drive and it's pretty horrible until OS X is fully loaded or updated; basically anything that uses the hard drive at the same time including MDS caching seems to slow EVERYTHING down a lot worse than the past (CPUs are doing nothing, but the disk activity holds it up). I'm sure SSDs don't have that issue due to random access speeds being so darn high, but I really think Apple damaged something that worked better before in that regard and perhaps didn't notice it since they use so many SSDs now by default (the new iMacs must be awful with conventional drives, though).
So my hypothesis is that the iTunes store is busy caching things to the drive while it's trying to play and that's causing an issue sooner or later. I did notice that MDS Worker had like 18 instances running in the background when it first happened. That's a lot of separate caching threads. I also noticed every time iTunes starts since V12.x it indexes or something EVERY SINGLE TIME even if you just quit and restarted. I don't remember iTunes behaving like that in the past at all and I've been using iTunes + AppleTV since the 1st gen model came out and ran a PowerPC Server back then with it and had no issues at all save a bad release of iTunes a couple of times (back then Apple was VERY responsive to fix iTunes crashes on the PPC models for me when I reported and actually got back to me; now they NEVER communicate for bug reports unless it's an iTunes billing issue). You'd think a company worth almost a TRILLION dollars could do a better job communicating with the public about bug reports and the like. Having people waste an hour on the phone with support trying things that can't/won't fix it when the problem is across so many machines is ridiculous, IMO. Charging for that is even worse.