It cannot be turned off.
I disagree with your analysis. I believe you can shield the iPhone from others and click the numbers quickly to prevent their view. Further, the glowing helps provide positive feedback to you that you clicked the right keys.
Also, you might enjoy the iPhone 5S which allows a fingerprint to replace the passcode.