Why does Apple TV says it is not authorized to play content sometimes when I use the iPad Remote?

Sometimes when using my Apple TV device with my TV off and trying to play music using the iPad Remote app, the app hangs and no music plays. When I turn on my TV to see what's going on there is a message suggesting the Apple TV is not authorized to play this content. Clearly, that is not the case and when I use the normal Apple TV remote to select the music, it plays fine. I use iTunes Match and generally stream my music from there on the Apple TV. I have both HDMI and optical connections to my receiver. Any idea why this could be happening? I would say it happens several times a week.