It is sad, that it is now marked as "Solved" as it is just a workaround, not a permanent solution. Apple needs to fix it.
I bought Mac mini M1 and I've experienced same problem just with Sony MDR-1000X headphones. It helped to turn WiFi Off, but this is totally unacceptable.
I tried to install 11.2 Beta 2 and RC to see whether the issue is fixed. However I noticed a significant improvement it was still choppy. I haven't run into this thread earlier, so Mac mini is just waiting to be picked up for a return to Apple.
Now I have MacBook Air M1 on my desk to test whether it suffer same problem, and unfortunately it does. I ran it for a while with original Mac OS 11.0.1 which was as quite OK, but still bit choppy. I updated to 11.1 and it became unusable. Each song (4-5minutes) was interrupted several times. Now I'm trying it with the workaround Location service off and it looks so far so good.
I still have Mac mini 2018 and it does not suffer of this problem, even with location service being turned on.
Anyway, lot of kudos to you, mrmm.one, for narrowing it down to Location services. Good job!