I can connect Spotify and other apps from my phone to my Apple TV, which is strange.
That seems to imply that the network connection is functional. Could it be that you are sending to an AirPlay-enabled TV, instead of to the Apple TV box?
it looks like the wifi is not connected (wifi symbol with exclamation point).
You mean in between the time and the account icon, in the upper right corner? Like this?
First try a router/access point restart, followed by an Apple TV box restart (disconnect from power for ~30 sec.).
If it doesn’t automagically connect again, then you will need a functional remote to first set the network settings, so that both devices are on the same home network. That’s why you would want to replace the missing remote, even if you prefer the software remote. There are emergency situations where the physical remote is functional, where the software remote isn’t.
Try to find an acquaintance to borrow a good Apple Remote from temporarily. Else buy a replacement remote. Replacement remotes will have to be paired.
The old style Apple Remote (aluminum) A1294 would work too for this. Network settings can be done without the need to press the TV button, that the old style remote lacks.
Once both the Apple TV box and the iPhone are on the same home network again, they should connect easily.
If you lost your Apple TV remote - Apple Support
Set up the Apple TV Remote on your iPhone or iPad - Apple Support
