I'm also having this problem. It seemed to have started after I allowed the Apple TV to update. Now I disconnect after a few minutes of audio playing, or airplay video playing.
Both the computer and appletv are up to date.
The apple TV is hard-wired to a GbE port. The laptop is a new Mavericks MBA on a 5GHz AC router. Zero interference because I live in the country.) It is the only device I used on 5GHz band. I use 2.4Ghz for my phones and all over devices in the home.
I'm happy to troubleshoot. I'm ordering a USB->GBe adaptor for the MBA to confirm it isn't wifi related. The new MBA's on Mavericks are confirmed to have an issue with wireless where packets can become very latent due potentially due to power saving mode being set in the driver. I have disabled this powersaving mode by running 5 pings a second to a site so I have sub 2ms latency to my router.
I've already escalated the wifi issue up the chain at Apple so if the ethernet solves the problem, I will add that to the wifi issues of the MBA. If it doesn't, I will start an issue with Apple over the ATV.