If you want the quad core , 1Tb of storage and 4GB RAM now go for it.
You are not forced to use the server app if you don't want too.
Just set it up as you would any standard Lion client.
The real considerations are that the server Mac Mini has the integrated graphics just as the base machine.
The 2.5gb i5 Machine comes with an AMD Radeon HD 6630M 256mb discrete graphic chip for much better graphic performance and allows you full use of that 4GB RAM for running apps and all at a lower cost.
If you can't get that locally buy from Apple on Line
While the server app is separate in the Mac App Store, on a new Mini it's already part of the OS and cannot be separated. I have the 2010 server Mini, and apart from the lack of Boot Camp (which I don't use on an HTPC) it performs the same as Lion client. What you do have to do is turn off, or not turn on during setup, all of the services that the server normally runs. You might also be able to make a Lion client boot USB stick (must be a later version/build than what came with the computer) and try to wipe the drive(s) and reinstall the client only.