I have two Apple TV2 and two Airplay Audio items in my network. Using Fritz Box 7490 as router.
I had the same problem. Also the four items were not always popping up completely in the Airplay menu of my iPhones. Very often, the Apple TVs were not popping up or one of them was not popping up.
Then the above problem (2) occurred in addition and not only this, but the streaming did not always work.
This is how I solved all this trouble:
I deleted all Airplay devices in my router, one by one, restarting the router after each single action. This is important, because after so many trials I got the impression, that the router remembers somehow old settings.
I renamed my Apple TVs in the disconnected state and chose completely new names that I had not used before !!!!
I connected them one by one and chose the standard "automatic" modes in their network settings.
When they showed up in the router, I told the router to apply always the same IP.
Thats all and it works fine now. All four items show up in the Airplay Menu with the correct names.