You can't. If you purchased 10.5 Server as a separate boxed installer the booklet that comes with it explains the 3 server modes/configuration options, what they mean and what your options are. If you didn't:
You can't reinstall the Server OS without reformatting the Server's Drive. As always try and keep as many current and effective backups as you can and never forget the three golden rules;- DNS, DNS and DNS.