Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

VPN to Mountain Lion Server issues

Hi,


I checked a lot of VPN threads here today, but I wasn't able to find a solution for my problem just now. I try to connect by VPN to my Mountain Lion Server, but I get an error message that the VPN server is not responding. I get this message from iPhone and Mac. The Mountain Lion Server is a new installation, no upgrade from an older server.


Some informations on my setup:

  • I installed the server with a hostname like myserver.mycompany.com and option 3 (internet access), as I want to use it for email at a later stage. All services are working fine (except VPN). DNS is active, but basically it only contains the adress myserver.mycompany.com and forwards everything else to our router.
  • I changed the DNS settings of our domain ( hosted by an ISP - so not in the local DNS ! ). I created a subdomain vpn.mycompany.com which points to the static IP of our router.
  • In the router I opened the UDP ports 500, 1701 and 4500, and for 1701 i made the same thing for TCP (I found this in a forum, but I think this is not necessary?), the ports are pointing to the ip of the os x server.
  • In OS X Server I started VPN for L2TP using the vpn.mycompany.com hostname, and a shared secret.


When I try to connect with I client from outside I try to connect using L2TP via vpn.mycompany.com using the shared secred and user-id and password. The user-id is created in OS X Mountain Lion server and is configured to use VPN service. When trying to connect I get the error message "L2TP-VPN server is not repsonding...".


In the log file of the server I see some entries for each connect:


Oct 10 20:21:45 myserver.mycompany.com racoon[13873]: Connecting.

Oct 10 20:21:45 myserver.mycompany.com racoon[13873]: IPSec Phase1 started (Initiated by peer).

Oct 10 20:21:45 myserver.mycompany.com racoon[13873]: IKE Packet: receive success. (Responder, Main-Mode message 1).

Oct 10 20:21:45 myserver.mycompany.com racoon[13873]: IKE Packet: transmit success. (Responder, Main-Mode message 2).

Oct 10 20:21:45 myserver.mycompany.com racoon[13873]: IKE Packet: receive success. (Responder, Main-Mode message 3).

Oct 10 20:21:45 myserver.mycompany.com racoon[13873]: IKE Packet: transmit success. (Responder, Main-Mode message 4).

Oct 10 20:21:48 myserver.mycompany.com racoon[13873]: IKE Packet: transmit success. (Phase1 Retransmit).

Oct 10 20:22:06 --- last message repeated 2 times ---

Oct 10 20:22:06 myserver.mycompany.com com.apple.SecurityServer[17]: Succeeded authorizing right 'system.privilege.admin' by client '/Applications/Server.app/Contents/ServerRoot/usr/libexec/ServerEventAgent' [2967] for authorization created by '/Applications/Server.app/Contents/ServerRoot/usr/libexec/ServerEventAgent' [2967] (2,0)

Oct 10 20:22:06 myserver.mycompany.com com.apple.SecurityServer[17]: Succeeded authorizing right 'system.privilege.admin' by client '/Library/PrivilegedHelperTools/com.apple.serverd' [1716] for authorization created by '/Applications/Server.app/Contents/ServerRoot/usr/libexec/ServerEventAgent' [2967] (100000,0)

Oct 10 20:22:06 myserver.mycompany.com racoon[13873]: IKE Packet: transmit success. (Phase1 Retransmit).

No more entries in log file now. Anyone any ideas what's going wrong. Might there be a problem as I use another servername outside as inside (vpn... instead of myserver...)?


Thanks!

Posted on Oct 10, 2012 12:08 PM

Reply
Question marked as Best reply

Posted on Oct 12, 2012 2:00 AM

Solved, first of all we tested to establish the VPN connection locally by adding the ip address of the server to /etc/hosts for vpn.mycompany.com. The VPN connected without problems then, so it was clear that it is a firewall/router problem, and not a server problem.


After that we studied some more documentations and found that we don't have to open port 50, but ip protocoll 50 (ESP) on the firewall. After that was done, the connection was working from the internet as well.

2 replies
Question marked as Best reply

Oct 12, 2012 2:00 AM in response to MccoQy

Solved, first of all we tested to establish the VPN connection locally by adding the ip address of the server to /etc/hosts for vpn.mycompany.com. The VPN connected without problems then, so it was clear that it is a firewall/router problem, and not a server problem.


After that we studied some more documentations and found that we don't have to open port 50, but ip protocoll 50 (ESP) on the firewall. After that was done, the connection was working from the internet as well.

Nov 13, 2012 9:38 AM in response to MccoQy

Hi MccoQy

I have the same problem and just one question regarding your solution. You say you opened ip protocoll 50 (ESP) on the firewall. Can you explain where you did this? Thanks in advance for your help

Venus

VPN to Mountain Lion Server issues

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple ID.