I had this problem both between Mac itunes library and Apple TV and Windows 7 Itunes Library to Apple TV, I disabled IPv6 on both my mac and windows machine, fixed the ip on all my devices. I also disabled windows firewall on my windows machine (I have an external firewall so not needed) you may not want to do this. From my windows machine it would play a few songs, then start one at 7 seconds and not play audio. From my mac it would say connecting to itunes library with the wheel.
These are the things I believe to resolve the issue, I also disabled a tonne of windows services (iphelper, Windows media streaming)