Believe it or not, unplugging your Apple TV, waiting 10 seconds, and plugging it back in can make a difference. Do this for every device connected to your Apple TV. I was having similar issues with my parents' Apple TV and I had to "reboot" their Vizio smart TV and their Apple TV and since then it has been working properly.
In any case here is something you can do to narrow down the problem:
Try downloading the Remote app for iPhone and pair it with your Apple TV. The Remote app speaks to your Apple TV over WiFi and will reveal whether you're having a Siri Remote Bluetooth issue. If the Remote app is working properly then you might have a problem with your Siri Remote. If the Remote app for iPhone is misbehaving also, then it's possible that you either have a WiFi issue or the Apple TV is misconfigured. In either case, you should reset your Apple TV via Settings -> System -> Reset and Update and begin setup from scratch ( yes I know :-( ). Once the system is reset to it's defaults verify that the Siri Remote is working properly with the App Store. If the problems persist, you should contact Apple Support or visit an Apple Store with your Apple TV and Siri Remote.
Please come back and let us know if and how you resolved the issue. Good luck!
