I have the same issue and I've decided to replace my iMac within the 2 week grace period when that option was offered to me through tech support.
As a note SMC and pRAM were reset during my call to Apple and the sound is still there. After downloading iStat Pro my fan speed was reading 1400rpm which appears to be the idle speed on the new 21.5 iMacs.
My concern isn't the volume as much as the nature of the sound which I would liken to a feint but audible high speed ticking. Put me in mind of an old 1.0 litre Fiat I once owned with the engine ticking over way too fast ...
Even though it's a very quiet sound, it's incredibly distracting when you're trying to work. Having said that, I imagine it would be almost inaudible in an office environment.
I have the 21.5" with a fusion drive and for a while I was worried that it might be the hard drive, but that fear was dispelled when I downloaded SMC fan control. When I upped the fan speed to 1500rpm and beyond the clicking/ticking sound got faster. Ergo, noisy fan.
Also note I was unable to reduce the fan speed below 1400.
Even though I'm taking advantage of my grace period I'm kind of resigned to the possibility that the replacement Mac will be the same. At least I know it's not the fusion drive but it's still kind of irritating.
However, I won't pre-judge, I'll update this thread as soon as I know, hopefully in the next few days when the replacement arrives.