Any difference if the Apple TV is on or off when connecting to reciever input one with the reciever set as HDMI input 1
Basically Apple TV disconnected but on or off and reciever set to port 1.
Repeat for port 2 as the result should be the same.
Obviously when it fails confirm that the switch back and forth solves the problem.
I have to take a step back and at least consider my own Apple TV, it is not working perfectly.
I got a brand new replacement at the store and both has the same issue.
The problem exists but you are pretty alone with it.
I have only briefly tested plugging my Apple TV directly into my speaker system as the handshake through my 2011 samsung home cinema 5.1 does not support 4K. No problem wth sound.
If I were to bet and you had a different Apple TV the problem would remain while if you tried another reciever the problem would probably disappear instantly. That is the sad truth.
There are far too few people experiencing your problem for it widespread and a handshake problem is hard to fix as a consumer. You might be one of very few with your problem.
Try to do the best of the situation.
I ran across an Apple TV official feedback form and in my oppinion the problem when you narrow it down should be reported directly to apple and not their apple community.
They read every feedback and reply sometimes if they are able to.
https://www.apple.com/feedback/appletv.html