To temporarily fix the problem (sound crazy) but in this case violence is the only answer up until you take a trip to the good people at Apple. By that i mean give it a good tap on your knee or gently squeeze under the volume button, it's not the best way but hey it does get annoying and can fry the battery and overheat things, what is wrong is a common defect in many iPhone 4s which is the backplate inside the phone is not seated correctly which allows it to some how turn on the flashlight.
Best bet is to take it into Apple or take off the back and re-seat the plate.
Hope this works as it did for mine, if the problem persists then they will gladly help you out or even giove you a replacement since it is a hardware fault and nothing you did to make it happen.
The service is alway phenominal, even when you have the smallest or biggest of problems they will always fix it with a great attitude and are happy to help.