iMac, Multiple Monitors & Spaces

I have 2 monitors connected to my new iMac. I arrange various setups using Spaces on all 3 monitors. When the iMac comes out of sleep mode, all of the panels are always pulled into the iMac monitor (e.g. the other monitors are empty) in each Space. How do I get it to stay "as is" after it comes out of sleep mode?