Resetting the SMC usually does the trick:
Resetting the SMC (System Management Controller) on Intel-based Macs:
Resetting your Mac's PRAM and NVRAM:
Firmware updates for Intel Macs:
First check the obvious like is anything plugged into the headphone jack. If not then recheck System Preferences and see if the sound is turned up and mute is NOT checked. If it's still OK then do a SMC reset, if you aren't familiar with that then please Google "Intel iMac SMC Reset" and you will find instructions. If for some reason that doesn't work restart in Safe Mode, this will clear some caches. It is possible a cache has become corrupt. To do a Safe Mode start when you hear the startup tone hold down the Shift Key until a progress bar appears, let it fully boot then restart normally and test.