That sounds rather frustrating... I can't recall how my iPhone was used during setup, but I believe it was used to simplify configuration with all the data stored in iCloud conected to my Apple ID (WiFi passcodes, etc.).
Have you already created an Apple ID and entered it into the Apple TV during setup? This page of the ATV User Guide, Set up your Apple ID on Apple TV, seems to imply that it's possible to get the job done without another Apple device.
Beyond setup, there's a spiffy Apple TV 4K color calibration process that uses an iPhone's front facing camera, but that's purely optional and I don't recall being nagged to do it.
Best of luck.