Can I connect apple watch to TV without iphone being connected to wi-fi.

When my phone is in the bedroom and out of the living room wifi range I get an error on the remote app on the watch that says "Connect to Wi-Fi. Make sure your iPhone is in range and connected to the same network as the device you want to control". Is there a way around that?

I would like to be able to control my apple TV without having to get up and get my phone otherwise it kind of defeats the purpose.