Hosting or Joining multiplayer Aracde games via GameCenter Join Codes fails from Apple TV 4K

This is a super frustrating one. Oddly enough I dont seem to see anyone else having run into it before.

Just signed up for Arcade so the family can play stuff like Marble Knights and Crossy Castle. I am a gaming snob so I am all about on a big TV with Xbox Controller, thank you Apple TV!

If we all play on our ipads it works fine. Problem only happens when I try to use the Apple TV.

From Apple TV, I can host a game and get a code but the ipads can not join, it just times out.

Also from the Apple TV, I can try to join a party code, same thing just times out.

Side Note: I can on Crossy Castle use wifi scanning however my kids accounts are restricted from this I guess since they dont have the option (which is also upsetting). But MK doesnt have this option.

I do need this party join code stuff to work on Apple TV to iPads (or joining a iPad hosted game via party code).

This must be a bug in Game Center? My guess is that party code is native to iOS via Game Center APIs. The problem must be in there. Other steps I have tried is full system restarts and logging out and back into the Apple TV.

More technical background: iPads are connected to two different SSIDs (one for 2.4Ghz and one for 5Ghz) while the Apple TV is on Ethernet. All three connections are on the same IP Subnet once connected with no broadcast restrictions. I have no aggressive firewall issues and have a 1GB Fiber based internet connection. iPads all are iOS 14, Apple TV was up to date with iOS14 but I have also tried beta 14.2 but that didn't fix it.