Volume (IR) no longer working on Siri remote

I have two 4th-gen AppleTVs. My living room setup suddenly stopped being able to control the volume on my receiver with the Siri remote. I tried fiddling with the settings, power cycling everything, unpair and re-pairing the remote and nothing is able to restore volume control functionality on the remote. I'm now certain that it is the remote itself after I swapped remotes between my living room and my bedroom where the other AppleTV is. After re-pairing, the bad remote is unable to control the volume on my bedroom TV.

I've searched around and there doesn't appear to be any way to do a hard reset of the Siri remote. You can't take the battery out to cycle it that way and it would take weeks for the battery to drain. I'm guessing the only thing to do is take the remote to an Apple Store for replacement.