One homepod in a pair is always responding, the other always silent

We have two homepods paired, one on the far end of our living room and the other next to our fridge in the kitchen. When we're in the kitchen (very close to the kitchen pod and very far from the living room one) and ask Siri to do something, lately it's always the one in the living room that responds. I've even gone up close to the one in the kitchen and given it a command very quietly and the living room one responds just the same. The kitchen homepod is clearly hearing us, but the living room takes over. This is super frustrating because when we're in the kitchen, the living room one can be hard to hear.

I've already unpaired and paired them multiple times as well as restarted them. What else can I do to change this annoying behavior?