Logic browser preview during playback is delayed

In Logic when I preview sounds in the browser window on the right side of the screen it will delay the playback until two beats later if the song is playing. How can I get this madness to stop? I can preview sounds quickly without delay if the song is not playing. This is literally killing my workflow and I'd hate to have to go back to Cubase because I can't do drums in Logic without wanting to snap my laptop in half!!