I think I kinda understand your question, so I'll give it a shot:
You can probably lean towards either a 2.1 system OR a nice pair of "near field" stereo powered speakers and be very satisfied. Technically, your iMac does have an optical output, so it will pass thru a 5.1 mix to your external AV system or self contained 5.1 system with an optical input.
But you really are not suffering with a nice pair of powered 'stereo' speakers or a nice 2.1 system connected via your headphone output (instead of trying to go 5.1) on your new iMac. I don't game much, so I've never "missed" having a 5.1 system at my homeoffice desk (and I prefer watching 5.1 movies on my THX home theatre). My 5-Year old compact RSL Digital Fidelity 2.1 system still rocks me from 6' away (with a little help from SRS iWow plugin). They fit so perfectly on my desks design.
But a set of reference monitors will be my next upgrade (I am in no hurry);
The M-Audio speaker line mentioned above (ie: BX-5a or AV40)) is a very good idea of what a modern pair of reference monitors, near field, looks like, and what kind of specs they have ('near-field'meaning that you will mostly be sitting near speakers, not trying to power a giant room.
Personally, I am going to see if I can figure out a high quality adaptor and look at the Mackie line.
I like the sound of it too: Mackie monitors for my Mac studio.
http://www.mackie.com/products/studiomonitors/index.html
I am really intrigued by the MR8 and MR5: excellent bang-for-buck reference monitors IMO.
http://www.youtube.com/watch?v=mpHL4IjbunY