Stuttering and distorted sound after a few seconds with Apple Watch 9 and AirPods Pro 2

Hi all! I found a few similar topics, but most have different errors or relate to older watch models.

So here is my problem:

My AirPods Pro 2 (updated to latest firmware) work perfectly with my MacBook Air and my iPhone 12 Pro Max.

They also connect without any trouble to my Watch 9.

However, when I start to play music from the watch, the sound is ok for 3-5 seconds, then there is massive stuttering, distorted sound, and lagging (sound continues for about 1 sec after I stop the music on watch).

I. have tried some of the solutions that were suggested in other topics, like pairing again, resetting the AirPods, switching other close-by devices to flight mode, switching off in-ear detection, etc. - to no avail.

Playing music from the watch DOES work with other non-Apple earbuds, those however are inconvenient because they can't switch between devices easily.

Any ideas what I could still try? Being able to leave the phone behind sometimes and listening to music with watch and AirPods was my main reason to get the AirPods Pro, and I would really love to get this working...

Thanks for suggestions!