Remote app looses connection to iTunes or won't connect

I have been trying to get my Remote app for iPhone to work with iTunes for months. I have gotten used to that it only works when it randomly wants to. Many times I see the library in the app but it won't connect (times out). Other times it connects and works somewhat for a while and then just looses connection randomly without any input from me, and I have to try to restart everything (router, PC, iPhone, sometimes one, sometimes all, it's all random) to get it working again. Internal IP adresses have been locked in router so they won't change.

I have no firewall in windows on, and have tried disabling IPv6 like someone suggested somewhere. No improvement. PC and iPhone are connected to same router, and are in the same network. PC uses cable, iPhone ofc WiFi. I have been in contact with Apple support, and they tried but couldn't help. Got a link to a home sharing troubleshooting guide afterwards, but there was nothing new for me there. I also reset the network settings in the iPhone, with no change. And reinstalled the network adapter in my PC, with no change. Tried disabling a few different services in Windows that affect iTunes and/or the network, without change.

Also, may be unrelated, may not be. iTunes sometimes become unresponsive when I try to close it after the Remote connection has gone. Takes forever for it to close. If I manually turn off home sharing first it seems to work better. If the app hasn't lost connection this freezing doesn't happen when I close iTunes.

I'm just about to give up on using the app. Even though it would be perfect for me, to use as a "what's playing" display and remote while gaming or otherwise using the PC for fullscreen stuff.