First, make sure the computer can "breathe." Clean the air intakes on the bottom edge of the case to remove dust. A lot can build up there. Then inspect the exhaust vent, the long slot running across the top of the back panel about 12mm (0.5 inch) below the top edge. Both the intake and exhaust ports must be unobstructed. If you have stacks of paper or othes object under the computer it will overheat. You need roughly 75mm (3 inches) of clearance around the air intakes and twice that between the exhaust port and any surface.
Second run Apple Hardware Tests:
How to use Apple Hardware Test on your Mac - Apple Support
If you get any error messages, post them. Otherwise simple say if AHT reports no issues.
Fans are thermostatically controlled so the fact that the fans come on after you start using it indicates that the heat sensors on the logic board are working as designed. We have the same model 2011 iMac here and it is virtually silent in operation.
There are software issues that can cause enough stress on the system to turn on teh fans. Anti-virus, so-called "cleaning/maintenance" apps, torrent software, etc can ru up the temps.