I tried a Mini once and sold it after 2 months because it was just too slow for me - but then it depends on what your needs are, of course. The Mini has the same type of drive as a laptop (5400 rpm), whereas a desktop has a 7200 rpm, which is obviously quite a bit faster. So, you'll need to think of what you will be using it for.
As for your 5,1 iMac - I pulled up the complete specs here:
http://www.everymac.com/systems/apple/imac/stats/imac-core-2-duo-2.16-20-inch-sp ecs.html
Since yours is what I had (except for a 24" screen), I thought I remembered that the *maximum RAM for that machine is 3 GB*, although it's been reported that there would be a slight advantage by going to 4 GB. It will only recognize a fraction above the 3 GB though - something like 3.2 or so. I had mine maxed at 3 GB (one 1 GB and one 2 GB module) and it was quite good, although much much slower than the new iMac I have now (see my profile).