I had this problem with my Macbook Pro 15" and the cause turned out to be a faulty power connector, the classic defect of the cable end starting to separate from the housing that clings to the Mac. I have a couple of adapters and was using an old MagSafe supply with the straight connector.
This issue was driving me nuts. When viewing the Displays control in System Preferences, even moving the Brightness slider to the full right position and unchecking the option to "Automatically adjust brightness" I watched the Brightness slider automatically do a slow, random "bounce" between the full position and a level about 20% less. It kept randomly gliding between the two levels. At first I thought something must be wrong with my hardware, like the light sensor was messed up and somehow still controlling things, or that the trackpad or keyboard was stuck. But it still happened even when the preference pane was closed. Virus? No! I then noticed the LED indicator going on and off on the MagSafe connector in sync with the shifting display levels. Thank goodness for that second setting tucked in Energy Saver on the Battery tab to "Slightly dim the display while on battery power" and the LED or I would not have noticed my battery discharging when I thought it was on AC power.
For now I wrapped some tape around the end where the cable enters the MagSafe housing, problem solved.