If you use external speakers connected to the headphones port, you need to be aware that Mac OS X keeps separate sound volume settings for every sound output option you have. Most people have two: +Internal Speakers+ and
Headphones.
The startup sound always uses the internal speakers and therefore uses the sound volume setting for +Internal Speakers+. The external speakers are using the sound volume setting for
Headphones. Therefore, no matter how low you set that volume (for
Headphones ), the sound volume of the internal speakers at startup will not be affected.
The solution (assuming what I described above applies to your setup) is simple. Disconnect the external speakers from the headphones jack. You are now using the internal speakers. Set the volume for the internal speakers to something really low. Reconnect the external speakers to the headphones jack. Your iMac is back to using the sound volume setting for
Headphones (automatically) but the really low setting for +Internal Speakers+ is retained. When you restart, it will use that new really low setting for +Internal Speakers+; when startup completes, the iMac is using the higher (normal) sound volume setting for
Headphones.
If that does not work, or you do not use external speakers, you can also try resetting PRAM
http://support.apple.com/kb/HT1379
The internal speaker volume setting is stored there. If it is somehow corrupted, resetting PRAM should resolve the problem. Then, reset the volume for +Internal Speakers+ and that volume should be used for the startup sound going forward.