Airplay video on iPad audio on TV when streaming from Home Share

If I stream video from my iTunes Library to my iPad, then turn on AirPlay to my ATV, the video plays on the iPad but audio plays on the TV. I could use the Apple TV to grab the content from my iTunes library, but why the weird behavior? Is there a fix?