Too slow full screen window switching. When switching I have to wait a second to type.

When I have 2 apps (say chrome and iTerm2) on full screen and I four-finger-swipe between the two, I cannot immediately write on iTerm2, it takes a second or two for it to be fully responsive. Is this expected? Is there a more efficient setup I can use? Any suggestions, even if they do not directly fix this, that offer an alternative solution are welcome!