Have you checked ARD on the computer you have configured to be the task server?
you could try turning off ipv6 if it's not used on your network; or at least local traffic only.
If you try starting the task on the task server it self dose it work?
if starting the task from the task server works, is the task server using a different SN key? (assuming your version of ard requires a SN)
also I've noticed that for my computer to use the task server I have to give it more then 6 computers for a task. Just and FYI kind of thing.