I have narrowed this issue down to the internet traffic levels on my setup. When I use my thunder bolt wired connection I don't have the problem. The problem only occurs when using heavy internet over my wireless connection.
MacBook Pro with Retina display, OS X Yosemite (10.10.3)