dem107, I'm in the same boat as well.
I have done everything you have mentioned. I think I might have narrowed it down a little further experimenting with gfxCardStatus. I personally believe it is related to the Intel unit, not the NVIDIA. Mine is also a mid-2012 15" retina (2.6 Ghz/16 GB/ 512 GB Samsung SSD). I can run it in the forced integrated graphics for approximately 4-5 days. This includes closing the lid to make it sleep. Typically, if I haven't rebooted at that point, I experience the system freeze. It has varied. Sometimes it will be when "waking up," but other times it will occur after 10-15 minutes of use.
When I completely disable the automatic switching and force it to the NVIDIA, I can run it weeks at a time without any issues. I know during the boot process the system will do a quick switch between the graphics cards, but I'm not sure if this occurs when the computer is "waking up" from "sleep" mode. I don't believe this occurs when the automatic switching is disabled. One issue I've run into is that my settings will not save (disabling automatic switching); once I've rebooted switching is re-activated. There was a similar issue with getting notifications to save when going to Yosemite, and it involved deleting a folder. If anyone has a fix, please let me know. (Thanks in advance!)
As an FYI, I run for work Windows 8.1 in Parallels 10 as well as many of the Adobe Creative Cloud Apps (natively in OS X) (I will often have to run Windows simultaneously with InDesign, Illustrator, Photoshop, Premiere, and SAP. I always have Outlook open and sometimes some of the Office programs.). The computer definitely gets a workout, and the 650M graphics card handles it all fine with no system freezes, as long as switching to the Intel doesn't occur.
My issues did not occur until Yosemite (just like the majority of users). I'm still leaning towards there being a software/firmware issue with Yosemite and the Intel unit. Like some of the other developers on here, I had also submitted this issue and it was closed as a repeat issue directing to the same submission others have received.
For those recommending to go back to old platforms such as Lion, that is just not feasible for many of us. The modern applications require the latests platforms. It would be like returning to Windows XP (well, I guess not as bad as that is just a security nightmare in today's world). At the same time, I hope that Cook can better address this as Jobs never used to put up with "buggy" releases like this. On the flip side, it is still significantly more stable than even the latest versions of Windows. My desktop workstation in my office is a Late 2013 27" iMac 3.5 Ghz i7/32 GB/1TB SSD/GTX 780M. That one runs Windows 8.1 in Boot Camp. I run the Adobe stuff on the OS X side and our SAP applications in Windows. Yosemite on there runs fine with no issues. So even though this switching thing is not excusable and very annoying (having to use workarounds), it could be worse. I hope they get this fixed soon to bring this back to the level of an Apple (especially with a $3,000 unit).