I realize it's been some time since doctored asked about upgrading, but I thought that my experience might be helpful. I've done almost everything you can do, and no, it was not fun.
rkaufmann87 makes an excellent point: if you are really interested in a much more performant computer, then by far the cheapest, easiest, and least risk approach is to sell your late 2006 iMac on e-bay and apply that to a newer used computer, or refurb, or whatever you like.
But I have found out first hand that the upgrades are not all that expensive and if you are looking for modest gains, they can be had.
The #1, easiest and possibly best upgrade is to max out your RAM. Get 2 matching 2G SODIMMs into the machine and watch it go faster. It's cheap, and only requires removing a small metal plate on the bottom of the computer. Many people can do this themselves. I consider this a no-brainer for Lion, and would recommend it for all users that do more than just e-mail and browse the web. For me it helped iPhoto and iTunes speed up a lot. If you have to replace both SODIMMs because the one[s] you have are not 2G, then it will cost you somewhere between $40-80.
The next biggest bang for the buck is the HD. Sadly, while the drive is cheap, the process is definitely not so easy. Opening the machine and removing the display requires some better than average mechanical and electrical skills, and there are delicate wires and connectors, including a heat sensor glued onto the existing HD. Unless you are very confident, I would not recommend opening the case on any Intel iMac. However, if you are skillful and want to do this, you can add a (say) 2TB drive that will perform better and hold more, or you can use an SSD, which will have less capacity, but be faster yet. The funny thing is that all this space will let you load up iPhoto and iTunes to the point you absolutely need that RAM I just mentioned. The 2TB HD will run ~$100 and a 256G SSD will run about $200 with the physical adapter.
The next step is the graphics card. Mine failed, so I purchased the 7600GT that CrankTurner mentioned. It was hard to find, hard to purchase, and really hard to install. I also upgraded the HD at that time because I had to open the machine anyway and *that* made the HD upgrade a no-brainer. In comparison the HD was easy. Unless you are a gamer, there is little benefit to gain and a lot of money to lose and what I consider more than a modest risk of ruining your whole computer. I did it because I had nothing to lose with a broken computer, and it took me several tries to install the new one and get it to work correctly. These can cost well over $300.
Lastly, if you didn't get the computer with the 2.33GHz CPU, you can upgrade to that. There is so little difference between that and the 2.16 you have, that (IMO) there is no reason to spend your money and take your chances on an CPU change. If you absolutely must, these are reliable components, so I would feel confident buying a used one from ebay for ~$80.
Finally, this machine will not run Mountain Lion without serious software hacking. It's my understanding that even with the upgraded video card it's still not powerful enough for Mountain Lion. I seem to recall that this generation GPU lacks some kind of video shader features needed for ML - so none of this will get you past Lion - but maybe a simple memory upgrade might make Lion work better for you.