If it's not working at all, the first thing to check is to make sure that port 5900 is open on the path between your admin system and the client, and that it's forwarded if necessary to the client's IP address. If it's working erratically, then you may have network problems or the client's IP address is shifting due to DHCP lease expiration or the system being restarted.
Without more details about your network between the admin system and client it's difficult to offer much more in the way of advice, but perhaps that will help.
Regards.