Post Mavericks (server) upgrade, vpn has stopped working. Any suggestions?

To clarify - I can connect internally just not externally. I've checked all the port forwarding and NAT settings on my router and even tried a different router - this all worked perfectly for over a year before upgrading to Maverics. In addition to this I eliminated my Airport all together and connected my imac directly to my cable modem which has a static external IP and set it to open and forward everything to my imac just for testing still wouldn't work. I'm going to keep trying things, I work for a large organization and in my job i work directly with some really smart network engineers so i'm going to see if they have any ideas.

As far as the apple support guy connecting I believe that only worked when he had me turn on PPTP. I show noting in my logs to see that he actually connected, He just told me he connected fine. Not sure I belive that.

My MB will now connect, iPhone (iOS7) and PC's are still not working. Very confusing

I am getting more and more confused with this problem. As I can connect to my VPN from one of my clients....see log below.

Thu Oct 24 12:36:12 2013 : L2TP incoming call in progress from 'XX.48.130.XX'...

Thu Oct 24 12:36:12 2013 : L2TP received SCCRQ

Thu Oct 24 12:36:12 2013 : L2TP sent SCCRP

Thu Oct 24 12:36:12 2013 : L2TP received SCCCN

Thu Oct 24 12:36:12 2013 : L2TP received ICRQ

Thu Oct 24 12:36:12 2013 : L2TP sent ICRP

Thu Oct 24 12:36:12 2013 : L2TP received ICCN

Thu Oct 24 12:36:12 2013 : L2TP connection established.

However I just tried to connect my iPhone to my Mavericks Server VPN and it will not work. Mind you that both my laptop and my iPhone are on the same Wi-Fi network at the client I am at. It does not show any connection requests from my iPhone on the log I watched it when I tried to connect.

I just dont understand it, It only works from this clients network it wont connect when I am using my AT&T LTE network my, my Offices Wi-Fi & Wired ethernet or inside my own network. What the heck??!!!

I'm have the same issues but I was able to get PPTP to work and this does appear to be a NAT issue. However, the NAT issue is sort of on the clients end and not the servers side. At work I'm on a 10.0.10.x network and I'm not able to connect from multiple systems. When I disconnect my iPad from the WiFi and go over the cell network with a routable address I can connect. I can also connect from my laptop when I have personal hotspot enabled on my iPad.

In both instances my IP address is 76.x.x.x but If I use a MiFi where my IP address is 192.x.x.x I'm not able to connect. I'm guessing that the server is seeing the internal IP address (10.0.10.x or 192.x.x.x) and is trying to route the return replies to that address instead of the NAT IP of the client (76.x.x.x when using my iPad's cell connection).

I'm going to call Apple and give them the information I have found to hopefully speed up the fix.

I have spoken to Apple Enterprise support and they are going to give the information to engineering. They said the fix would probably be in a form of a patch but had no ETA.

I can confirm if you change you setting to allow PPTP connections VPN will work correctly. It appears that the more secure connection method of L2TP is broken on Server 3.0. I will continue to research and find out if there is some work around for L2TP connections as it worked for me twice both from one of my clients.

I was using L2TP with shared secret and after upgrading to Mavericks it was also not working for my windows vpn client. After a lot of messing around and trying parameters, it seems that setting "Negotiate multi-link for single link connections" in my VPN connection properties under Options/PPP Settings has made the difference. Data encryption on the security tab is set to Require encryption and I've got CHAP and CHAP v2 enabled for authentication protocals.

Still can't seem to get my iphone to connect to it however, though I'm pretty sure I had off and on luck with it the day after the upgrade...

For what it's worth, my VPN started working after I rebooted my server. I had been having timeouts with Mavericks Server, as well. Things seem good for now. The VPN service in OS X Server has been wonky for me ever since Snow Leopard Server. I have at times had to restart the VPN after I reboot the server. I am going to hang onto my clone that still has Mountain Lion Server for now, though.

Same here. from with the local network it works. Airport extreme was automatically changed bij installing maverick server, breaking my PPTP to my NAS (I had L2TP to os x server and PPTP to NAS, just to be sure I can enter if server needs reset; I found on ML that I had to reset VPN server sometimes)

So I took the tcp 1723 port out of the forwarding to teh Maverick server and now the PPTP VPN to the NAS works again.

The L2TP tVPN to the maverick server only works from the local network, with proper logs. coming from the outside, no logs in the server and an error mesag on my macbook air that the server does not respond. I kept all the UDP port in the forwarding to teh Maverick server (500, 1701, 4500)

With ML this worked just fine. what's up Apple?

Pascal

confirmed L2TP only working on LAN when using local IP of the server. Does not work on LAN using hostname resolved with dyndns client. And does not work on from external network, again using dyndns resolved hostname .. Have confirmed dyndns is resolving correctly, and have also tried connecting using my actual WAN IP as apposed to dyndns hostname.

Mavericks Server 3 running on MacMini server, Router is Airport extreme, client is macBook Pro running Mavericks.

NB - L2TP between Mac Mini Server running 10.8.5 and Server App and MacBook Pro running 10.9 worked just fine (Why did I trust in Apple releasing a fully functional Server App for 10.9 I dont now!!). Also I have another L2TP server I connect to, also working fine from MBP running 10.9!!

Honestly Apple - Brand loyalty only goes so far you bunch of 'effing clowns!!

Your's - Extremely Disappointedly - FKA

Also tried brianfromround lake 's suggestion of using Hotspot from my iPhone, but this didn't work for me!

Also have turned on PPTP (Not that I want to open my network like this!!) and it does not work - 'Authentication Failed' ..

FKA

OK no Open Directory setup, so no PPTP access - but either way I don't want to use PPTP - It's simply no way near as secure as L2TP ..

*ALL TOgther NOW* "why are we waiting, oh why are we waiting ..... zzZZZ"

OpenVPN here we go again ... It works, it's stable. Just means I'm going to have to cough up for VMWare 6 because, guess what? My old VM 4 its totes fooked with mavericks ...

I used to look forward to CrApple updates, new features, new look ... now a just think Bah, should have stuck with the last version ... The only, and i mean ONLY, reason I'm not rolling back to ML - Quicker SMB transfer !

I love my Windows Home Server 2011 ! It does what it says on the tin!

I spent 4 hours on the phone with Apple Enterprise Support (kept getting transferred up the support person chain). At one point we created a new account for the support person and they tried the VPN, which worked. At that point he told me if must be my connections on the other devices I was trying and that was all support could do- he also told me that when pinging my server he was seeing a lot of packet loss. After this I requested that he guide me through completely uninstalling the server, since every time I do this it carries over settings (right down to the shared secret) when reinstalling. He wanted to verify that we were doing it correctly so he put me on hold- at this point I ran a web based ping and traceroute (to rule out my local network) coming from Europe NO packet loss. We did a complete uninstall of the server portion and reinstall and the shared secret came back- which he could not explain. I then showed him the ping/traceroute- which he insisted since he could connect (which we could see in the logs) that it was my local networks for the other machines I was testing on, also that this was as far as support could go. For instance trying my iPhone on AT&T's LTE network and a work laptop- VPN into the work network THEN trying to come back to my VPN network. I informed him that I was going to completely nuke the machine and start over- he advised that I try the VPN from a Starbucks first, still insisting that it was my local cell/work networks being spotty.

I erased the main drive, re-installed Mavericks, then re-installed server 3 annnnnnnnd I am back to the same place. Currently I am testing from another location and still cannot get past the IKE Phase 1 portion of racoon's auth via hostname HOWEVER if I use Logmein to get back to another machine on the same network and use the IP I can use the VPN.

Here's the log:

Oct 23 08:22:10 hostname racoon[224]: Connecting.

Oct 23 08:22:10 hostname racoon[224]: IPSec Phase 1 started (Initiated by peer).

Oct 23 08:22:10 hostname racoon[224]: IKE Packet: receive success. (Responder, Main-Mode message 1).

Oct 23 08:22:10 hostname racoon[224]: >>>>> phase change status = Phase 1 started by us

Oct 23 08:22:10 hostname racoon[224]: IKE Packet: transmit success. (Responder, Main-Mode message 2).

Oct 23 08:22:10 hostname racoon[224]: IKE Packet: receive success. (Responder, Main-Mode message 3).

Oct 23 08:22:10 hostname racoon[224]: IKE Packet: transmit success. (Responder, Main-Mode message 4).

Oct 23 08:22:10 hostname racoon[224]: Connecting.

Oct 23 08:22:14 hostname racoon[224]: IKE Packet: transmit success. (Phase 1 Retransmit).

Oct 23 08:22:47 --- last message repeated 3 times ---

Oct 23 08:22:50 hostname racoon[224]: IKE Packet: transmit success. (Phase 1 Retransmit).

Oct 23 08:23:10 hostname racoon[224]: IKE Packet: transmit success. (Phase 1 Retransmit).

Oct 23 08:23:59 --- last message repeated 1 time ---

Oct 23 08:23:59 hostname racoon[224]: IKE Packet: transmit success. (Phase 1 Retransmit).

Oct 23 08:24:56 --- last message repeated 1 time ---

Oct 23 08:24:59 hostname racoon[224]: IKEv1 Phase 1: maximum retransmits. (Phase 1 Maximum Retransmits).

Oct 23 08:24:59 hostname racoon[224]: Phase 1 negotiation failed due to time up. 2194c11c97819d97:a29d73f04fe7e67f

The L2TP-VPN server did not respond. Try reconnecting. If the problem continues, verify your settings and contact your Administrator.

Here’s the network topology- Internet > Modem > Airport Extreme > Mac Mini via ethernet.

For the hostname DNS we are using a dynamic DNS service, which I have verified is resolving to the machine through the router ect.

I have tried deleting the Server App and /Library/Server as well as any pref files I could find, then rebooting, after downloading the Server App again I found all of my settings are back. Also I’ve tried removing the Server Setup Done file as well in conjunction as well as independently with no luck.

I have tried killing raccoon via the activity monitor as well as via the command line.

I am able to reach the machine locally via ssh and screen share, and externally via logmein.

I have tried an iPhone 5s locally and externally, and two MacBook Airs internally and externally as well.

Tried messing with racoon's access to private keys still no luck. Currently trying to restore a TM back in a VM to run on the machine.

I have deleted the VPN port forwarding entry in the Airport, tried putting it back manually as well as via the Server App and the drop down menu in the Airport.

I am 99% sure the traffic is reaching the server as I can see the following when I try to authenticate to the VPN, please note this is always the same for each VPN client

So at this point I am stuck either rolling back to ML or getting on the phone again with Enterprise Support who is convinced that it's not on them.

I erased the main drive, re-installed Mavericks, then re-installed server 3 annnnnnnnd I am back to the same place. Currently I am testing from another location and still cannot get past the IKE Phase 1 portion of racoon's auth via hostname HOWEVER if I use Logmein to get back to another machine on the same network and use the IP I can use the VPN.

I have another call with support tomorrow morning as I am starting to see message boards and App Store reviews saying the same thing- Server 3.0 seems to have broke the already fragile OS X VPN Server...