ARD quality is bad with one connection, but clears up with two.

I'm having a new issue that surfaced after I upgraded from Yosemite 10.10.3 to 10.10.4 last month. I have a 2009 27.1" iMac at my home that has no issues. However, if I connect to it via ARD at my office, the quality is very poor and slow, and prior to the upgrade it was fine. Today I discovered that if I connect the home iMac using ARD on two different machines at the office at the same time, the connection clears up. This issue also occurs regardless of where I am connecting from (work, Starbucks, friends house).

Any thoughts why this would be happening? Why would just one ARD connection be poor, but two simultaneous connects are good?!