Homesharing: iTunes to Apple TV stream is extremely slow

I've noticed that streaming any video (music, movie, tv, podcast) from my iTunes to my Apple TV is running extremely slow, where I'll start up a movie and come back 1/2 hour later to see that 5 minutes has buffered and the movie hasn't started playing.

Netflix, Hulu, Apple purchases from the cloud stream fine, but none of my content on my iTunes.

Anyone got a fix for this or should I call Apple and start screaming at them about this?