AirPods low volume on one side with Apple Watch

Seems like if I put in one AirPod at a time on and use music on my Apple Watch, on AirPod is always really low in volume. Even if I take it out and then back in. I have to take both out, and then put both in at the same time for it to work. I can replicate this pretty often. When I place one in and then the other it tends to occur, then taking both out and placing them on at the exact same time fixes the volume issue. It mostly happens with my Apple Watch, and I know it’s not dirt or the AirPods because it happens with the AirPods and AirPods Pro. They can’t both have the same issue, so I’m guessing it’s something with the Apple Watch. I keep the AirPods really clean and have done the whole unpair and repair thing.

Anyone else experience this?