I have found a reliable way to reproduce the problem quickly for troubleshooting, and have relayed to Apple support.
My MacMini is set to auto login, and starts iTunes on login.
1. Boot up Mac, start iTunes
2. Switch on Apple TV
3. Stream a movie from Apple TV for a minute or so
-- so far, so good --
4. Stop playing movie, Sleep AppleTV (General menu > Sleep)
5. Sleep Mac (Apple icon -> Sleep)
Wait for the Mac to sleep, the white light on front pulses on and off
Don't touch the mac from this point onward
-- wait a few minutes --
6. Wake AppleTV
7. Hold breath in anticipation!
8. Select Computers ...... it shows up and I can play a movie, as it should.
* note - the mac wakes, but the monitor attached to the Mac stays blank *
-- still ok up to this point --
9. Stop movie playing, Sleep Apple TV
11. Wait 10 minutes - the Mac decides to go to sleep, irrispective of the Sleep settings (mine is set to 20 minutes by the way, and the display sleep is set to 5 minutes.)
Now,
12. Wake AppleTV
13. Select Computers
Problem: The Mac iTunes library shows up, but on attempting to select 'Movies' you get a spinning wheel on the TV screen for about 2 monutes before it gives up.
The Mac does not wake up!
14. Hit the spacebar on the MacMini ... it wakes, everything springs back into life.
Go back to step 3, it all happens the same way.
Conclusions?
After running network sniffers, and dns-sd commands (there is also a nice iPhone app called 'Bonjour Discovery' that shows the status of home_sharing - I see that the sleep_proxy broadcasts stop being sent from the Mac once it has put itself to sleep.
If the Mac sleeps because of no user activity (the first time) then it will wake again, however if the Mac wakes into 'dark mode' and then sleeps, then it falls off the network.
I found that if I set the boot options for 'darkwake=0' and repeat the above test, then at step 8 above the monitor springs into life, and the Mac now works as it should do - i.e. it does not disappear off the network and it continues to wake / sleep - basically work properly.
I found that if I set my sleep settings too high - above 50 minutes, then things start going wrong again, and the mac stops waking. 15 minutes was the optimum for me, but not less than 10 or it does not work.
This has worked for me for 5 days so far, I did have one hic-cup, and found there to be a problem with iTunes - another long story, but switching off iTunes Match, De-authorising the computer, and signing out of iTunes, then signing back in, and switching back on did the trick.
The main thing I don't like about this solution, is that the Mac wakes up once an hour through the night, drives spin up, screen comes on - it associates with the network, then dozes off again after 15 minutes.
So although I have a solution, I'm not happy with the long term effects of spinning up the drives once an hour - all through the night 24/7.
I ended up just setting the Mac to stay on all the time, gave up on the wake on network option, and set it to wake at 8am, and sleep at 1am .... this will have to do until Apple sort this annoying problem out.
I have a MacMini 2012, OSX10.8.2, iTunes 11.0.1 (latest at this point in time.)
2012 Time Capsule, and several Apple TV-3 - all connected over WiFI on 5GHz network band. Router is a BT Infinity modem (I did away with the BT Home Hub and plugged the Time Capsule directly into the modem.)
In my testing I also took a 2008 MacBookPro and wiped it and updated to Mountain Lion 10.8.2 - it shoed exactly the same faults - I also factory reset the Time Capsule and Apple TV, and signed them all back in, and switched on Home Sharing and iTunes Match.
One annoyance was that the MBP2008 kept bugging me to allow the firewall every time I started iTunes. I wiped the machine again and reinstalled - I discovered that the last update corrupts iTunes 😟 - codesign -v iTunes.app reports it is corrupt, and I had to delete iTunes manually, re-download from Apple and it worked - a bit disappointing really.
Finally, I have a new 2012 MacBookPro Retina - same issues.
I've tried all the great suggestions in this thread, and found the above (darkwake=0) and Sleep=15 to be the best I could get, but not good enough for me.
If you got to the end of my post, I'd be interested in knowing if anyone else gets the same results, that is if your patience has not given out with rather annoying issue.
And for what it's worth, Snow Leopard works fine, I reinstalled this on the MBP2008, it only goes bad from Lion (in my tests.)