Slow Response Time on Dual Display

I just got a Thunderbolt Display to replace a cheap poor-resolution display and I had the idea that I could hook them both up to my Mac Pro (Late 2013) at the same time. I had understood that this model could run several displays.

Well, they are connected but the Thunderbolt, which I set up as the main display, seems very slow. If I drag an icon, for example, it takes literally about three seconds for it to disappear from one place and appear in the other. It is also very difficult to drag icons from one display to the other, even though the workspaces are arranged next to each other.

The Thunderbolt Display is attached to the Mac with Thunderbolt, obviously. The second monitor is using HDMI.

Any thoughts? I'm running the latest Yosemite.