13 Replies Latest reply: Feb 14, 2014 6:31 AM by FredrikHedman
johnsenisi Level 1 Level 1 (0 points)

What is the most common reason that I can connect to the newly created vpn locally but not remotely (over the wan)?


OS X Mountain Lion (10.8.4)
  • 1. Re: set up vpn on mac osx server
    bfdulock Level 2 Level 2 (160 points)

    The most likely reason is a misconfiguration or non-configuration of the router/firewall.  By that I mean the proper ports are not open or forwarded correctly.

     

    Assuming you are using an OS X Server, the following table shows what ports should be open/forwarded on your router for VPN to work from outside the local network:

     

     

    FOR VPN USING L2TP over IPSec

    --------------------------------------------------------------------------

    UDP 500    -->  internal server address

    UDP 1701  -->  internal server address

    UDP 4500  -->  internal server address

     

    IP-ESP (IP protocol 50, ESP)     passthru enabled

     

     

     

    FOR VPN USING PPTP

    --------------------------------------------------------------------------

    TCP 1723    -->  internal server address

     

    IP-GRE (IP protocol 47)     passthru enabled

     

     

     

    *** See for further information:

    http://support.apple.com/kb/TS1629

     

     

     

    Bryan Dulock

    Houston, TX

  • 2. Re: set up vpn on mac osx server
    bfdulock Level 2 Level 2 (160 points)

    Another reason could be an incorrect IP address.

     

     

    Bryan Dulock

    Houston, TX

  • 3. Re: set up vpn on mac osx server
    johnsenisi Level 1 Level 1 (0 points)

    I checked all settings, everything is correct per Apple's directions. The connection is allowed on the LAN. There is no WAN connection. VPN is turned on so the tunnel is available. Airport Base Station has the correct ports enabled (OSX Server did this for me in the set up process). I am at a loss as to why a connection outside the LAN is not possible.

  • 4. Re: set up vpn on mac osx server
    bfdulock Level 2 Level 2 (160 points)

    Questions:

     

    - when connecting over the WAN, is your computer on an outside network?

    - when connecting over the WAN, are you using the public IP of your router?

    - is the AirPort Base Station the actual router?

    - what does the VPN log on the server say?

  • 5. Re: set up vpn on mac osx server
    johnsenisi Level 1 Level 1 (0 points)

    I'm testing it thru an ipad personal hotspot, which would be outside the local area network.

     

    No, i have tried to use the public address of the router with no luck.

     

    Airport is the router, ahead of it is a Time Warner cable modem

     

    Usually there is no connection at all so the log does not show anything

  • 6. Re: set up vpn on mac osx server
    bfdulock Level 2 Level 2 (160 points)

    Is the Time Warner modem configured in bridge mode?  If not, the TW modem is acting as the router and the AirPort Extreme would be superfluous as a router.  This would also explain why VPN from the outside is not working.

  • 7. Re: set up vpn on mac osx server
    johnsenisi Level 1 Level 1 (0 points)

    I have previously looked into the interface of the Time Warner modem and did not notice one way or the other if it is set to "bridge" mode, or even if I have any control over it. I will check again and get back with what I find.

  • 8. Re: set up vpn on mac osx server
    bfdulock Level 2 Level 2 (160 points)

    Another possibility, it the TW modem is acting as your router, is the put the AirPort Extreme in bridge mode and just use the TW modem as your router.  Sometimes that is a simpler setup.

  • 9. Re: set up vpn on mac osx server
    bfdulock Level 2 Level 2 (160 points)

    Sorry about the typos above.  Here's a rewrite:

     

    Another possibility, assuming the TW modem is already acting as a router, is to leave it as the router and put the AirPort Extreme in bridge mode.  Sometimes this is a simpler setup.

  • 10. Re: set up vpn on mac osx server
    johnsenisi Level 1 Level 1 (0 points)

    I bought the new cable modem, the Time Warner leased modem had no functionality (and there is a monthly cost).

     

    I set up Apple Base Station to bridge mode.

     

    Internet functionality is back with new Zoom cable modem dishing out the ip addresses, etc.

     

    However, now the only way I can connect on the LAN is by typing the explicit ip addess of the server in the VPN configuration screen. Using the VPN Host Name will not connect.

     

    I don't even dare trying the WAN until I get the LAn working correctly.

     

    Any ideas?

  • 11. Re: set up vpn on mac osx server
    Oakleef Level 1 Level 1 (0 points)

    Hi.

     

    I'm having similar problems, but with the difference that I can't access VPN over LAN nor WAN. If I change the address that I'm connecting to, to 10.0.1.21 which ofc is my internal IP, it works, but when I type my external IP address or vpn.mydomain.com it doesn't.

     

    For portforwarding I've got 500, 1701, 4500 for UDP and 1723 for TCP.

     

    Any ideas? I'm drawing blanks here. Tried almost everything... Except the thing that makes it work.

     

    My websitedomain is forwarded to my IP from my host and that works just fine, but VPN won't...

  • 12. Re: set up vpn on mac osx server
    techboss Level 1 Level 1 (0 points)

    Same problems with L2TP server on Mavericks after upgrade.

     

    I turned on logging level to in /etc/racoon.conf:

     

    # "log" specifies logging level.  It is followed by either "notify", "debug"

    # or "debug2".

    log debug2;

     

    Restarted vpnd and tried to connect and then checked out my console.  I saw the connection attempt being made and all looked ok up until it got stuck hitting numerous:

     

    10/30/13 11:28:13.544 PM racoon[348]: Malformed cookie received or the spi expired.

     

    errors then finally logging:

     

    10/30/13 11:30:08.964 PM racoon[348]: Resend Phase 1 packet e9d548d778586159:6eae3dbe3fe45f70

    10/30/13 11:30:41.961 PM racoon[348]: IKEv1 Phase 1: maximum retransmits. (Phase 1 Maximum Retransmits).

    10/30/13 11:30:41.961 PM racoon[348]: Phase 1 negotiation failed due to time up. e9d548d778586159:6eae3dbe3fe45f70

    10/30/13 11:30:41.961 PM racoon[348]: Disconnecting. (Connection tried to negotiate for, 169.760629 seconds).

    com.apple.message.domain: com.apple.Networking.ipsec.connect.plain

    com.apple.message.result: failure

    com.apple.message.signature: Phase 1 negotiation failed (Maximum retransmits). NAT detected by Me

    com.apple.message.value2: 169.760629

     

     

    10/30/13 11:30:41.961 PM racoon[348]: IKE Phase 1 Failure-Rate Statistic. (Failure-Rate = 100.000).

    com.apple.message.domain: com.apple.Networking.ipsec.phasestats.plain

    com.apple.message.result: noop

    com.apple.message.signature: IKE Phase 1 Failure-Rate Statistic

    com.apple.message.value: 100.000

     

     

    10/30/13 11:30:41.961 PM racoon[348]: Freeing IKE-Session to 166.147.119.217[60405].

    10/30/13 11:30:41.961 PM racoon[348]: IV freed

     

     

    Out of gas for the night trying to track this down...anyone else care to try to move the discussion forward?

  • 13. Re: set up vpn on mac osx server
    FredrikHedman Level 1 Level 1 (0 points)

    I have had the same problem with setting up L2TP on Mavericks after upgrade.  After several failed attempts I have the following recepie.

     

    To test this you need to have two separate networks to connect you VPN client to.  One should be the same as where the server is running and the other needs to be different so that the incoming traffic to your router is coming from the outside.

     

    I'm assuming a setup with a router and behind it a local network with an OS X server running the VPN service (vpnd daemon)

     

    On the server

    • Note the local ip-adress of your server.  This should preferably be static.
    • Install the VPN fix from apple: http://support.apple.com/kb/DL1716
    • In the OS X Server VPN Service create a VPN profile where VPN Host Name is local ip-adress of the VPN server.
    • Restart the VPN service and save the configuration file.

     

    On the router

    • Open ports 500, 1701 and 4500 to pass UDP traffic to the server.  Make sure to activate them in the router interface.
    • Make a note of your routers public IP address. This should be static.
    • If this keeps changing you can set up a dynamic domain name (http://dyndns.org)
    • Optional: verify that the ports are actually open using nmap:

     

    sudo nmap -Pn -sU XX.XX.XX.XX -p500,1701,4500
    Password:
    
    Starting Nmap 6.40 ( http://nmap.org ) at 2014-02-14 14:21 CET
    Nmap scan report for ... (XX.XX.XX.XX)
    Host is up (0.012s latency).
    PORT     STATE         SERVICE
    500/udp  open          isakmp
    1701/udp open|filtered L2TP
    4500/udp open|filtered nat-t-ike
    
    Nmap done: 1 IP address (1 host up) scanned in 1.29 seconds
    

     

    • XX.XX.XX.XX is the public IP-adress of the router.  You can also try the same on the local IP-address of the server.

     

     

    On the client

    • Copy the configuration file and install it by double klicking on the file.
    • Connect the client to the same local network as the vpn-server and activate the VPN connection. 
      • Verify that the VPN connection comes up.
      • Up to this point, smooth sailing. 
    • Now change the Server address to the IP-address of the router and turn on extra logging found under Advanced. Save the new configuration.
    • Bring up the VPN connection again.  Should work.  Right?
      • I did not for me.  The error complains about the L2TP-VPN-server not responding.
      • Digging deeper using the system logger I found the error
    2014-02-14 14:43:31,039 racoon[60284]: IKE Packet: receive failed. (Malformed or unexpected cookie).
    2014-02-14 14:43:31,039 racoon[60284]: Malformed cookie received or the initiator's cookies collide.
    2014-02-14 14:43:31,172 pppd[60283]: IPSec connection failed
    2014-02-14 14:43:31,172 racoon[60284]: vpn_control socket closed by peer.
    2014-02-14 14:43:31,173 racoon[60284]: received disconnect all command
    
      • So it sort of works, but complains about some bad cookie.
      • The simple change of the IP-address apparentely generates this error.
    • Now change the network of the client so that it is not on the same networks as the server.
    • Bring up the VPN again.  Now it just works.
    • So apparently, when the traffic is coming in from the outside the VPN connection just works.
      • If you change back to the local network of the server and the keep the router IP-address the error is back.

    Conclusion

    • The conclusion is that the client used for connecting to the VPN network must be on an outside network.
    • In retrospect, this makes sense since we should test using an environment that reproduces the actual use case. The crux is to ensure that the client traffic is coming in from the outside.

     

    Hope this helps.