Yes, do Tlix's thing. And while you're at it, un-check "Start up automatically after a power failure." I can't see why anyone would want that to happen unless they're running their iMac as a web server.
But it's ALSO possible/likely that a Bluetooth device (your wireless keyboard or mouse) is waking it up. Go to System Preferences > Bluetooth > Advanced... and un-check the box beside "Allow Bluetooth devices to wake this computer."
Note: this DOES mean that you won't be able to wake your computer up from sleep by clicking the mouse or pressing a key on the keyboard. To wake your computer up from sleep, just single-press (as in, DON'T hold) the power button on the back of your iMac. It'll wake right up.
Good luck!