I have the same setup, and I have never had to “turn off” the Wi-Fi manually.
I did however have the same issue with sound, and it may be a few things.
1) Make sure you have an HDMI cord that is good enough to transfer 4K and above easily.
2) On your Apple TV, (under settings), try to recalibrate your audio to see if that fixes anything. (That fixed the issue of my wireless speakers lagging a bit behind the Front speakers - which are wire connected to Apple TV - which is plugged into Ethernet).
3) Last option I would try would be manually offsetting the audio on your sound system/TV. (Usually only need to do this if you are using any type of soundbar, or sound system, etc.)
You can then fine-tune the audio offset a bit ahead or behind in order to match the rest of the sound.
Hope these helped!