If the Control Center sleep timer wasn’t set, and no-one else in the vicinity has a remote to shut it off, then the only other things that I can think of, is overheating or a power issue. Could it be that the Apple TV box gets very hot, is insulated by a confined space, or is not on a flat surface (there is a fan at the bottom that needs air circulation). Could it be that the AC power gets interrupted? Small devices have small capacitors that may react to small (short) interruptions.
Certain apps may quit streaming for no interaction (“Are you still watching?”), and then the sleep setting for not streaming anything may kick in, but I suppose you would realize that, if such were the case.