Airplay 2 Sync Issue

Hi, I apologize if this has been discussed but I didn't see a thread on it. I'm having sync issues between my 2 Apple TVs (4k model) and other airplay 2 targets, including my 2 HomePods. Essentially, the HomePods and other speakers are late compared to the Apple TVs sound. In other words, the Apple TV plays the sound first and then it plays on the speakers/homepods.

I've run the wireless audio sync on both Apple TVs and it didn't really fix anything. I suspect this has to do with what I've read about airplay 2 - that there's an inherent lag built into it due to buffering. That's consistent if I try to airplay from my mac. There's a second or two between when I hit play/pause and when it actually happens.

But I guess I don't understand why the ATVs don't also sync up with this. I called apple support and they suggested me play from my Mac on both the HomePod and the internal speakers. I did that and they were in sync, so the issue does seem to be the appleTVs. I've run them through my TV and optical audio out and I've also connected them directly via HDMI to my soundbar. Doesn't seem to make a difference.

It doesn't seem to make a difference if I start the content on the TV and then airplay to the other speakers or vice-versa. The only thing that seems to actually work is if I start playing Video on the ATV, then select a HomePod as a target speaker, then reselect TV Speakers. Then it's in sync. But If I then immediately switch over to the Music app in on the ATV, it's back to having a delay

I've power cycled my whole home network and it didn't do anything to address the problem. The ATV's are all hardwired wired (via ethernet switches) that then connect back to the router.

Any ideas? I'm at a loss and feel like I've tried everything I can think of