"I don't really have a private network at the Co-Lo."
Just configure one of the interfaces with any private IP network/subnetmask and use that for your LAN or if you have more public IPs to spare, use just one interface for all connections: main IP, alias IP and VPN IP(s).
"I share a cabinet with an associate. We have a Router connected to a Hardware Firewall, connected to a switch, connected to our various servers."
A router and you have a 255.255.0.0 netmask on your side of the router???
What ports/protocols are open for your IPs in the hardware firewall???
"I have 2 IP addresses on one server, with both interface cards connected to the same subnet."
This is wrong. Use only one interface for this setup (and maybe the other for a private IP LAN).
"I really only need encrypted traffic to and from this one server... "
If you had 3 or more public IPs this would be easier. Otherwise you need to assign a private IP (-network) and NAT and configure the firewall (needed in both scenarios if you want security) in OS X Server.
"It is not like I am trying to access other machines on the "network" through the VPN... Being able to access the internet via the VPN tunnel is just extra... (and I will probably set up a routing definition so that most of this external traffic is not encrypted...)"
A VPN "private" routing definition does one thing: tells your VPN client what IP-packets to put in the VPN tunnel. Traffic is only encrypted between your VPN client computer and the VPN server, then it's unencrypted when leaving the server. It's only encrypted in the tunnel.
A VPN "public" routing definition (default route 0.0.0.0/0.0.0.0) is for allowing for simultaneous local Internet access through the VPN-client local Internet connection a.k.a Split Tunnel setup.
No VPN routing definition(s) - "force" sending the lot through the tunnel, the default is through the tunnel.
"I have just gotta get traffic to the server to be encrypted.
Oh yeah... the other issue is that the colo is in California and I am in Washington... so changing the way the server is connected is not really an easy option... and since all of the IPs we have from the ISP are on the same subnet... well Perhaps this just will not work."
It will work if setup properly, but I don't understand the router, network subnetmask, firewall configuration.
"But... Since my settings changes last night, I am able to encrypt traffic to
everything but my server?? Why is this??"
I don't really understand what you have working right now, but you need another IP to conect to the server's services when the tunnel is up. If you try the "main" IP (en0 interface?) the packets will be sent "straight on" - not through the tunnel.
A minimum configuration without the OS X firewall (to explain things???)
------------------------------------------------------------
all PUBLIC IPs on the same subnet:
main IP on en0 (subnetmask, router filed and DNS fileds filled in)
alias IP on en0 (only subnetmask filled in besides IP)
(An alias IP is when you configure a second IP on the same interface.
In Network config you can add a second "alias" interface for en0)
free IP for vpn client (for VPN config)
no routing def. (only DNS IPs)
NAT OFF but ipforwarding ON (this is for accessing Internet through the tunnel, alter /etc/hostconfig to say IPFORWARDING=-YES- , turns it on after a reboot, there are other ways to do this but if NAT and firewall is ON it's automatic).
en1 - disabled
You connect the tunnel to the main IP
The VPN client gets the "free" IP
The alias IP is what you use to connect to the server services through the VPN tunnel.
For protection you have to configure the firewall (if the hardware firewall isn't configured to do this).
If the hardware firewall was configured to let through the ESP protocol and UDP ports 500, 1701, 4500 to the main IP and block the rest, you could get to everything through the VPN tunnel using the server alias IP and access Internet through the tunnel (provided the hardware firewall lets the VPN client IP through -> Internet).
Any questions?