Phone calls on iPhone with Apple TV

I'm starting a FaceTime call on my iPhone. Pushing the speaker button enables me to transfer audio to my Mac or Apple TV.

My Apple TV is connected via HDMI to my flatscreen tv. Soundsticks are plugged into the tv's mini jack connector.

When I choose to transfer the audio to the Apple TV when I make a call, I do not hear anything.

A few seconds later, the sound is back on the iPhone. It'd be nice to leave the iPhone on the table and hear the caller through Apple TV. Any advice on how to do that?

Thanks!

Yannick