Windows jump off screen with external monitor

I have an external monitor connected to my MacBook Air. When I turn off the external monitor (MacBook lid is already closed at this point), and return hours later to turn on the monitor, I find most of my windows have been scattered (jumped) off the screen. It happens on several applications: terminal, iMail, Chrome browser, Preview, etc... I usually find the window peeking out of the side of the screen - a small sliver of the window - and I have to drag it back to the center of the screen. I have to do this for almost every window (again not all windows do this). Its very annoying. Any ideas about what's causing this?

I'm running El Capitan v10.11.1 (15B42) on MacBook Air (13-inch, Mid 2013). External monitor is Dell.