OSX Server for Mountain Lion is just an additional service.
For installing it, you moved "Server.app" in your Application folder and then run it.
For removing it you get some software like "AppCleaner".
You can download it HERE.
All you do now, move the "Server.app" to the "AppCleaner".
AppCleaner will list now all the files and folders that Server.app installed.
Restart your computer after you're done. There are most likely additional folder and configurations
but the main 'server' part should be gone. For everything else, look there:
http://support.apple.com/kb/HT4827