Two annoyances that I've found with Apple TV are that you can't sort the photo library by descending date order so to get to your most recent photos you have to scroll and scroll. In my case I have 13000 photos in my iPhoto library so it takes quite a while to reach the most recent photos I've taken. It seems such a simple thing that Apple can change to make a big difference to the Apple TV experience for me (and I'm sure many others).
The second annoyance is the inability to skip music tracks when watching a photo slideshow. The only way to currently do this is to stop and restart the slideshow and hope that the track that starts playing is one you want to listen to. I really don't understand why Apple removed this feature as it was always possible to do this with my 1st gen Apple TV.
Please fix these things Apple and make me a happier Apple TV user.