isn't the SMC powered by a battery? Is it possible the battery has died?
I don't think the SMC is but the PRAM is. There is a PRAM battery; it's a commonly available CR2032 Lithium button battery and its lifetime is around five years. It might be weak or dying. But I don't know if it directly controls the fan behavior. It might be related more to your initial question concerning erratic power on. This article lists what it does control.
http://support.apple.com/kb/ht1379
Not sure if this is your model, but just to give you an idea what might be involved in replacing it.
http://www.ifixit.com/Guide/Repair/Installing-iMac-Intel-20-Inch-EMC-2105-and-21 18-PRAM-Battery/1082/4
If it's just the CPU fan, it might be a bad sensor, but the Hardware Test, since you ran that 3 times, should have reported that. If you don't already have it or something like it (and you probably do, since you know it's only the CPU fan) I'd get free iStat Pro, so you can see the temps and fan speeds. Although, you seem to be saying the CPU fan is always running fast, even at startup, so that doesn't sound temperature related.
http://www.islayer.com/apps/istatpro/
If you can't get to the bottom of this on your own, I think it might be wise to bring it in to an Apple Store and let them run their diagnostics which are far more sophisticated than the Hardware Test; the Hardware Test might be missing something. I don't think they charge for that, but ask first to be sure.