This is my first post on here, so bare with me, and I hope this makes sense.
SOLUTION FOUND! (for Mac OS X Only).
I know how frustrating this is, and have been looking for a solution for a long time!
So after much trial and error, and eliminating possibilities - the following is what worked for me.
I have had the exact same issue with iTunes being unresponsive.
Running El Capitan 10.11.1 & iTunes 12.3.1.
The exact issue being that iTunes takes a while to load, and then whenever a song is playing, the next track will play, but the artwork does not display for what seems like ages (spinning beach ball and app freezes during this).
The CPU goes crazy above 100% in Activity Monitor and anything GPU related seems to struggle. I even had the whole screen freeze with pixelated distorted horizontal lines at one time! This initially made me think I may have a GPU hardware issue. But that would be extremely unlucky, considering this is happening on my [saved for ages!] brand new Retina 27” i7 Skylake Late 2015 iMac maxed out with 1TB of Flash!
However, I still have my old 2011 iMac to hand (not sold yet). I cloned my drive back to that, AND the SAME issue occurs (even more slowly). This confirmed to me that it was a software issue. Searching online, I found this forum and this also confirmed it has to be software, as you have all had issues around El Capitan and the latest iTunes.
So - is it an El Capitan or iTunes issue? Well, kind of both. Clearly there is some kind of clash between iTunes 12.3.1 and El Capitan. Ironic in that the install notes it says iTunes 12 has been built to work with El Capitan.
Anyway, I took the approach someone else mentioned here, and downgraded one step back to iTunes 12.3.0.44. (One step at a time).
THIS DID WORK FOR ME!
HOWEVER - the trick is to do it in a certain way to make sure it works properly. One step missed or the wrong way, and it may not work.
Clearly there is something in the iTunes 12.3.1 update that is causing this RIDICULOUS issue.
Apple REALLY NEED TO FIX THIS with the next update. Funny how it says that it improves overall stability and performance when in fact it does the opposite.
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::
The method below are the exact steps I took (link where needed).
BEFORE DOING ANYTHING - PLEASE BACK UP!
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::
I will not skip anything here, incase anyone reads this and is not so tech savvy. So please, tech people - don’t hate me for being over explanatory!
This is pretty painless, but a bit fiddly. No need to remove your carefully made Music Folder, etc.
Because iTunes is now protected from being deleted under El Capitan, you need to uninstall iTunes 12.3.1 in the following way;
01 : Locate iTunes in the Applications folder, click on it and press ‘Cmd key + i key’ for the info panel.
02 : Click on the padlock at the bottom of the window that appears (type in Administrator Password if needed to unlock).
Now change all the permissions to Read & Write. Re-lock and close the window.
03 : Now you can drag the iTunes app into the trash and delete it.
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::
Now you need to locate the following files to delete;
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::
04 : Click the ‘Go’ menu in the Finder — hold down the ‘Alt (Option)’ key — you will see the hidden ‘Library’ appear in the list.
While still holding ‘Alt’, click on it.
05 : In the window that appears, scroll down and double click into the iTunes folder in that list. Delete anything that is not in a folder.
DO NOT DELETE any of the following if you have them — iTunes Plug-ins, iPhone Carrier Support, iPad / iPhone Software Updates.
06 : Navigate back to the main ‘Library’ list (back button above). Now scroll down and double click on the ‘Preferences’ folder in the list.
Select ALL files with iTunes in them;
e.g.
com.apple.iTunes.eq.plist
com.apple.iTunes.plist
com.apple.iTunesHelper.plist
Drag them into the trash (type in Administrator Password if needed) and delete. Close the Library window.
07 : Go back to the ‘Go’ menu and select ‘Utilities’ near the bottom of the list and click to open. Double click on ‘Activity Monitor’ in that list.
Click the ‘Process Name’ Tab to put things in Alphabetical order. (This helps you find things easier).
08 : Scroll down the list, find and select the ‘iTunesHelper’ Application. Go to the ‘X’ button above to Quit the process.
If it doesn’t Quit, then just select Force Quit (it WILL Quit then!). Once done, you can now close the window.
09 : NOW you need to go into your System Preferences. Select ‘Users & Groups’. Select the ‘Current User’ on the left (usually the Administrator / Admin). Under ‘Login Items’ on the right tab, select the ‘iTunesHelper’ Application in the list. Click the minus button underneath to delete it from the list.
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::
After you have done all this, YOU MUST RESTART your Mac!
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::
10: After you’ve restarted, you need to download iTunes 12.3 (12.3.0.44) from the following link:
http://www.iphonefirmware.com/download-old-versions-of-itunes/
Select iTunes Mac OS X from the list, and in the list that appears, iTunes 12.3 is 2nd from the top. Click the arrow button to start the download.
11: Once downloaded, you can double click the dmg file to open, and just double click the ‘Install iTunes’ icon and follow the instructions.
12: Once installed. Close all open apps and do another RESTART.
13: Once you have restarted, open iTunes. YOU SHOULD NOW HAVE A MUCH BETTER WORKING ITUNES!!!
IMPORTANT : YOU MUST TURN OFF AUTOMATIC UPDATES - otherwise iTunes will update again to 12.3.1
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::
It still may have a bit of lag initially, but you shouldn’t have any hanging and spinning beach ball at all, or not for more than a few moments on occasion. You will need to ‘Sign In’ to your iTunes Account again. But only if you need to download things, etc. Doing so should not cause a problem.
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::
I hope this fixes things for some of you (or all)?
I’d love to know if this has helped :-)