Most thieves are smart & know to immediately turn an iPhone off & then restore it as a new device in iTunes. Doing this defeats Find My Phone. Find My Phone was never designed as theft protection or as a way to track stolen phones. While it can be used for such, its real function is to track lost iPhones. Of course, for it to work, Find My Phone must be turned on and activated, and the phone must have an Internet connection.
since I can track my ipad2 wifi 3g which currently dont have a sim card I would say yes
if gps is turned off or find my iphone app deleted or the thief restore the iphone to factory default
you will not be able to track your iphone
even if apple could turn gps on remotely or track it in secret they would be voilating the laws of most of the world and would suffer from persecution with extreme prejudice