I have iMac with Retina with 4GB video and 15" Retina MacBook Pro. Naturally I wanted to use the laptop as a second monitor. I tried Air Display (US$19.99) over ethernet and it does "work" but the entire system is too laggy to use. Mouse and keyboard response times are affected even when working only on the native iMac display. The virtual display's response time is even worse. This is true regardless of the settings chosen for the displays and for the Air Display Host app, and even when the virtual display is shown in a local window when the laptop is disconnected.
I suspect this may be because the iMac's graphics hardware is not strong enough to render 5k plus 2560x1600. Air Display can't help that but it might improve the situation if there were an option to reduce the virtual screen's resolution to something much smaller like 1024x640 without HiDPI. As it stands today, Air Display is useless for connecting a Retina iMac to a Retina MBP.