I would be concerned too if my Apple TV was not responding when using the remote. I found an article with some steps you can take when you experience an issue like this:
If you can see a picture but Apple TV isn't responding
Note: If the Apple TV status light does not flash white when you press a remote button, the issue may be with the remote. In this case, go to the the "If the remote isn't working" section below.
- Press and hold Menu on the Apple Remote to return to the Apple TV main menu.
- Reset Apple TV by unplugging it from the power outlet, waiting about five seconds, then plugging it in again or by pressing and holding Menu and down / scroll on your Apple remote.
- Power off your TV, wait five seconds, then power it back on.
- Make sure your TV is functioning correctly. See the documentation that came with your TV for more information.
You can find the full article here:
Thank you for using Apple Support Communities.