I have the same problem as you all above. I have an appointment at the Genius bar to return my rMBP today.
I have seen this at home,on both an older Netgear 802.11bg wifi router, and now a brand-new ASUS 802.11ac router. Alternatively, a 2007 Santa Rosa MBP, 2006 Intel Mini, iPhone4, and iPad2 all do just fine on both boxes.
On the Asus, which is dual-band, the rMBP does fine when it is on a 5 GHz channel, but as soon as it frequency hops to any channel on 2.4 GHz, things go to h*ll in a handbasket. The most I can do is stream low-bitrate audio - I can't do any file transfers, an playing even a standard-definition iTunes video makes the connection drop.
I love the rest of the machine, and I NEED the rest of the machine! This worries me greatly that it appears to be a systematic problem - I don't know if a replacment is going to be any better..... 8^(