Apple TV 4K remote delay issues

I purchased two new Apple TV 4K devices and have noticed an issue with BOTH remotes having delays. Have charged them both, but issues still remain. Sometimes it is a delay of several seconds from button push to response. Anyone else had this problem?