Hmmm... That may be the case on a few older ones; and maybe newer
hardware. I've tried a fairly recent third-party for-Apple extended keyboard
with power key and I get the same dialog box with that, I get when I press
the Control Key and the Eject Key together. So, I can shut down from it.
But powering up, as you said, will vary due to keyboard/model compatibility.
In some of these iMac G3 and G4 computers, if someone has been inside
them and didn't pay attention to detail(s) the power button can be misaligned
or damaged, and the button may stick due to friction against the case. If
that is evidently not the issue, it is best to avoid the problematic unit...
Unless it is free and you want to ebay the parts, to buy something else.
Since you can get an iMacG4 from places such as wegenermedia for as
little as (or less than) $400 for the basic unit, check for best working price.