Safari in iOS 6 keeps asking to use current location when searching

Almost every time I do a search in Google in Safari on my iPhone 5 or my iPad 2, which both have the newest version of iOS 6 installed, I get a message saying that google would like to use my current location. I will click on OK and then browse and go to a page. Then if I hit the back button to go back to the search results I get the message again. And if I do another search I most likely get the message again.

It is a real pain having to click OK every time. It never used to do this and I cannot pinpoint when it started but it was recently. Shouldn't it remember my choice when I click OK and always use my location?

Any ideas on how to stop this message from coming up?