My AppleTV 4 remote touchpad isn't working

I've tried restarting the ATV, updating the remote firmware, unpairing and pairing the remote, nothing works. For some reason, if I let the screen dimm and then swipe on the touchpad surface, it brings the screen back to life. So, I think there might be a software problem. No scroll in any direction. All the other buttons work just fine. Any ideas?