AppleTV 4k - making a mess of the HDMI-CEC signalling - flip/flop of sources

Starting a new topic since the one from November (4 months or so ago) is deemed too old to reply to.

Have a Sony XBR8 TV, and an older receiver without HDMI, so I have an HDMI switch box that has 4 HDMI inputs and one HDMI out with TOSlink optical audio out to the receiver, while the HDMI out goes to the TV.

The HDMI switch as on its inputs:

1) TivoHD

2) AppleTV (gen 2), now AppleTV 4k (gen 5)

3) XBox 360

4) XBox One X

(the game systems don't really matter, but wanted to just be complete on the description).

So the old AppleTV lived ok in the ecosystem. Its all controlled via a programmable remote control that plays macros when I press some buttons (such as ON or OFF or change source). The OFF macro shuts off the TV, which then via CEC powered down the HDMI switch. Similarly the HDMI switch would get powered on by the TV. The older AppleTV was powered on by the IR remote signal, not via HDMI. Things worked - except of course the older AppleTV had been abandoned for upgrades, so the podcast and streaming features would constantly hang up and not cache feed after watching 1 or 2 podcasts, so I'd have to constantly reboot the AppleTV in a watching session. PITA. So I upgraded to the newest AppleTV 4k. And things went nuts.

Right off the bat at the initial setup of the AppleTV 4k I saw things were screwed up. All I did was unplug the old AppleTV (HDMI cable, RJ45 network cable, Power cable), put the new AppleTV 4k in its place, and plug those 3 back in. On boot up of the AppleTV 4k the HDMI switch as manifested on the TV and Receiver flip/flopping between two HDMI sources: the TivoHD and the AppleTV 4K. The display and audio would toggle between the two nonstop - 1-2 seconds on one source, 1-2 seconds on the other, over and over, and over. After several minutes it would finally stabilize and stay on one of the sources. Got the setup done. But after a power reset, the same thing happened again. The only thing that changes it the AppleTV gen2 -> AppleTV gen5 upgrade.

I've changed the HDMI CEC settings in the AppleTV 4k to do my best to keep the AppleTV gen 5 silent (Just shut up and let ME control my inputs), and had told the TV to also not do anything (and then unplugged it so it would forget it ever saw a controlling AppleTV 4k gen5 device). But then the issue came back - and I think all I did was change the TV setting to have it power off/on other devices . Nothing in the AppleTV 4k gen 5 settings was set to re-enable HDMI-CEC services.

So as it is I cannot trust the AppleTV4k to just shut-up and let me control my system via the IR remote as I've always done. Is anyone else having this sort of issue? I may just have to return the AppleTV 4k gen 5 if this mayhem cannot be stabilized.