You have a 6 year old machine that you want to upgrade, to be honest it would be considerably wiser to invest in a refurbished 21.5 or 27" iMac and get a 1 year warranty that's eligible for AppleCare and a vastly superior machine. You can buy a refurb 27" currently in the US for about $1419, to upgrade your current machine would probably cost $500-$700 dollar to upgrade the RAM and HD alone (assuming you took it in to be professionally done) and you'd still have a 6 year old iMac with no warranty. Yes it would less expensive but IMHO a waste of money.
Your iMac supports 3GB > iMac (Late 2006): Memory Specifications, plus you can gain a slight inprovement by using a matching 4GB kit.
Yes a 4GB kit, see > Understanding Intel Mac RAM - Mac Guides
Other than that, your best bet is to replace it with a newer model.
WOW! worst advice on here. First off, that 06 iMac (model A1200 I assume?) can be heavily upgraded. Sure, it won't match the specs of a current model, but its fun to upgrade, and nowhere near the cost of a new one. You could start with an SSD, say a 256gb intel or ocz brand, Amazon has those for about $100-150. Next, definitely get a 4GB kit (your mac WILL support 4GB max), no more than $30-40 on Amazon. Then things get interesting, and here is where it CAN get pricey. You can upgrade the CPU on this model to a 2.33 or I believe a 2.5/2,6GHz core 2 duo processor (SMC fan comes into play here, to assist with cooling. Free app, easy to use). Used these run about $30-100, all depends on the seller honestly, but all are always in working condition and usually carry a minimum of a 30 day warranty, but usually 90 days - 1 year. The last, and most expensive option, is to upgrade the graphics chip. These have MXM type I or II video cards. If you have the 7300GT 128mb nvidia chip, you can upgrade to the 7600GT 256mb chip which is a huge upgrade believe it or not (SMC fan again). Sure it's an older card, but it will play just about any DX 9/10 game you throw at it and play it at full HD. Depending on where you go, sites like ebay have them for $80-200+. So at this point, you're at the $500-+ mark. If you are like me and love taking things apart, you can take your time with each upgrade and buy them as you're able to. Depending on how you use the machine, 1 or all of these upgrades can be super beneficial and totally extend the usability and life of the mac. Finally, those iMacs had the 2.16GHz core2duo cpu, so it will run Mountain Lion beautifully. Just my 2cents!
I realize it's been some time since doctored asked about upgrading, but I thought that my experience might be helpful. I've done almost everything you can do, and no, it was not fun.
rkaufmann87 makes an excellent point: if you are really interested in a much more performant computer, then by far the cheapest, easiest, and least risk approach is to sell your late 2006 iMac on e-bay and apply that to a newer used computer, or refurb, or whatever you like.
But I have found out first hand that the upgrades are not all that expensive and if you are looking for modest gains, they can be had.
The #1, easiest and possibly best upgrade is to max out your RAM. Get 2 matching 2G SODIMMs into the machine and watch it go faster. It's cheap, and only requires removing a small metal plate on the bottom of the computer. Many people can do this themselves. I consider this a no-brainer for Lion, and would recommend it for all users that do more than just e-mail and browse the web. For me it helped iPhoto and iTunes speed up a lot. If you have to replace both SODIMMs because the one[s] you have are not 2G, then it will cost you somewhere between $40-80.
The next biggest bang for the buck is the HD. Sadly, while the drive is cheap, the process is definitely not so easy. Opening the machine and removing the display requires some better than average mechanical and electrical skills, and there are delicate wires and connectors, including a heat sensor glued onto the existing HD. Unless you are very confident, I would not recommend opening the case on any Intel iMac. However, if you are skillful and want to do this, you can add a (say) 2TB drive that will perform better and hold more, or you can use an SSD, which will have less capacity, but be faster yet. The funny thing is that all this space will let you load up iPhoto and iTunes to the point you absolutely need that RAM I just mentioned. The 2TB HD will run ~$100 and a 256G SSD will run about $200 with the physical adapter.
The next step is the graphics card. Mine failed, so I purchased the 7600GT that CrankTurner mentioned. It was hard to find, hard to purchase, and really hard to install. I also upgraded the HD at that time because I had to open the machine anyway and *that* made the HD upgrade a no-brainer. In comparison the HD was easy. Unless you are a gamer, there is little benefit to gain and a lot of money to lose and what I consider more than a modest risk of ruining your whole computer. I did it because I had nothing to lose with a broken computer, and it took me several tries to install the new one and get it to work correctly. These can cost well over $300.
Lastly, if you didn't get the computer with the 2.33GHz CPU, you can upgrade to that. There is so little difference between that and the 2.16 you have, that (IMO) there is no reason to spend your money and take your chances on an CPU change. If you absolutely must, these are reliable components, so I would feel confident buying a used one from ebay for ~$80.
Finally, this machine will not run Mountain Lion without serious software hacking. It's my understanding that even with the upgraded video card it's still not powerful enough for Mountain Lion. I seem to recall that this generation GPU lacks some kind of video shader features needed for ML - so none of this will get you past Lion - but maybe a simple memory upgrade might make Lion work better for you.