One of my Airpods Gen 3 makes a buzzing noise

I got a new pair of airpods 3 the other day, and even though the left one works fine, the right one constantly makes a quiet buzzing noise. The noise isn't so loud that I can't drown it out with music, but it is there even if I'm not listening to anything and it's enough to be annoying. I've tried resetting them and reconnecting them with my phone, but nothing has worked to get rid of it. I'm hoping I won't have to go to the apple store to fix it but I'm not sure what else I should try first.