iPhone 5 headphones click/pop

My iPhone 5's headphones have a lot of clicking/popping whenever the sound mutes or unmutes. It is very noticeable when the keyboard turns on or off but the phone is otherwise silent.

I assume that it's due to poor offset trim on the headphone amps. Does everyone's 5 do this, or is it better in some phones and worse in others? I just got this phone today, and the sound is making me a little crazy. None of my other iPhones were close to being this bad.

I'd hate to have to swap phones, but this really bothers me a lot.