You can make a difference in the Apple Support Community!

When you sign up with your Apple Account, you can provide valuable feedback to other community members by upvoting helpful replies and User Tips.

Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

Sparty28 Author

Level 1

0 points

OS X Server / VPN /The L2TP-VPN server did not respond...HELP!

I am very new to OS X Server and my goal is to setup DNS & VPN! I would like to have this setup to be able to connect into my apple computer from work or friends house. I am using an Apple Airport Extreme router and im also using the latest version OS X Mountain Lion with OS X Server installed. I have started an account with dyndns website for user host name (using a _____@dyndns.org address). I assume this would be used as an alternate way of being able to connect without starting a personal website. I also signed up for another site (no-ip) and I now have a different IP address (not sure if that was necessary). I then followed instructions on youtube (instructional videos by todd for OS X Server Mountain Lion) which seemed to be very easy to understand. But after setting up my VPN on the client side (network setting in system preferences), i tried to connect VPN (L2TP) and i receive this error message "The L2TP-VPN server did not respond. Try reconnecting. If the problem continues, verify your settings and contact your Administrator.". When I open Consol in the utilities folder, I am seeing part of the following message below;

racoon[117]: IKE Packet: transmit success. (Phase1 Retransmit).

racoon[117]: IKE Packet: receive failed. (malformed or unexpected cookie).

pppd[490]: IPSec connection failed

Does anyone know what's happening or what I need to do to fix this? Or can someone tell me the basic requirements to setting things up correctly?

OS X Mountain Lion (10.8.2)

Posted on Mar 6, 2013 9:03 PM

Top-ranking reply

cpragman

Level 2

465 points

Posted on Mar 7, 2013 2:03 AM

Two things you didn't mention that I suggest you try:

1. at your firewall (router, modem, whatever), you need to open the ports used by the VPN protocol you are using. When the remote computer tries to connect to your server, your router has to pass the info from these port(s) on o your server so your server sees the incoming communication. Youll need to forward any incoming info on those particular ports on to your server's INTERNAL IP address (ex., 192.168.1.xx). Also, you'd want to set your server's internet prefs to have a static IP address inside your network. That way, it doesn't grab a randomly assigned IP address every time it boots up. If your server's internal IP address changes often, then your router can't forward traffic to it.

2. The address of your server will be <server>.dyndns.org, not <server>@dyndns.org. Note the dot, not the @.

View in context

19 replies

cpragman

Level 2

465 points

Mar 11, 2013 9:22 AM in response to Sparty28

Proxy shouldn't have anything to do with it.

Google search of your cable modem tells me that there's nothing to configure with your particular cable modem. It appears it will let all traffic thru.

Looks like you set up your AEBS to properly forward incoming VPN packets to your server. Check.

Got your server assigned to the address that the AEBS is forwarding packets to. Check.

Make sure you know your true external IP address. Some web pages will tell you this. Also DynDns updater utility will tell you this. That would be the address you'd use to contact your server when you are calling from outside your own network.

Sparty28 Author

Level 1

0 points

Mar 11, 2013 9:58 PM in response to cpragman

Yeah im not really sure what im doing wrong...i think i have tried everything possible. I google search my IP address and looked at my IP from noip.com and dyndns.com. Im using the correct IP 98.255.xx.xx, i have no idea what im missing or why i cant get this to work.

May 28, 2013 1:58 AM in response to Sparty28

cpragman, followed your instructions and it worked great, I saved the configuration file from the server app and pushed it to my IPhone 5. Install went great. The only thing I had to do was change the last digets in the IP Address in the VPN Config section from the router to the last digets of the IPv4. After that it connected. Now i can connect outside my netword with my MacBook Pro and my IPhone 5. Now to hit the IPad. Thanks for your help.

rsannie

Level 1

19 points

May 30, 2013 5:30 PM in response to Sparty28

I was having the same problem and I found out that there were two things I needed to do:

1) On the VPN Client (such as the iPhone), make sure you're entering the account name as displayed on your OSX Server (goto Users -> Edit User -- to check the account name):

OSX Server User info screenshot:

2) Make sure you turn OFF Back to My Mac (or remove the AppleID) via AirPort utility. See image below (should be empty). I was not able to log on to my VPN server if I have Back to My Mac enabled.

AirPort Utility Screenshot:

OS X Server / VPN /The L2TP-VPN server did not respond...HELP!