I have suffered with this problem for ages. The only thing that helps me is from Settings > Videos > Home Sharing > Sign Out, then Sign back in.
In my experience this problem absolutely is an issue with iOS. Some people with older software/hardware combinations (such as original iPad not upgraded) can use home sharing with no problem. I can use Apple Remote and third-party apps and access my admittedly large library with no slowdown, but I have given up hope that Apple will ever fix the problem with the Home Sharing over the Music app on iOS. I have done all the updates, spent countless hours on the phone with Apple Support, searched the Internet for a fix, all to no avail. I now use Mobi to play folders of my favorites from my Mac Mini and use Remote if I simply want to play to an AirPlay speaker. I wish you all the best, but I would not lose sleep over this issue thinking it's something with your setup or your library--the fault is completely Apple's.