10.9.3 update stops VPN access to Server on Mac Mini

Question

10.9.3 update stops VPN access to Server on Mac Mini

Having finally had the L2TP VPN issues solved after joining the 10.9.1 beta program for Mavericks and getting VPN access to our Mac Mini Server running again, the 10.9.3 update has broken it once more. It's been working since 10.9.1 and through both 10.9.2 and the 10.9.3 beta program, but after installing the final 10.9.3 update (without changing any settings on the Server App) last night it broke (immediatley after using the VPN to cheekily watch iPlayer abroad, so it was certainly working!) - now comes up with 'Authentication Failed'.

This happens on iOS devices as well, and all have authentication details stored (though naturally I have since tried recreating VPN configurations from scratch) so doesn't appear to be client end.

MacBook Pro with Retina display, OS X Mavericks (10.9)

Posted on May 17, 2014 5:58 AM

Reply

Answer 1

May 17, 2014 10:16 AM in response to Chris Billett

Surely, this wasn't the news I expected to see. I missed the 10.9.1 patch and was eagerly waiting the 10.9.3 upgrade. So, it seems that the upgrade is in conflict with the earlier patch. I was looking forward to complete my VPN setup later...this VPN is such a nightmare, and it's not just Apple.

Reply

Answer 2

burnduck

Level 1

0 points

May 18, 2014 4:00 AM in response to Chris Billett

Same here, although the issue is slightly different.

Updated Mac Mini with server.app to 10.9.3, L2TP VPN still works my mac running 10.9.3, connects as normal.

However other clients (windows and android) would encounter error when trying to establish connection to server.

Windows client would fail wirh Error 789, previous to the update it was working.

May 18 18:58:13 mms.private racoon[413]: IPSec Phase 1 started (Initiated by peer).

May 18 18:58:13 mms.private racoon[413]: invalid DH group 20.

May 18 18:58:13 mms.private racoon[413]: invalid DH group 19.

May 18 18:58:13 mms.private racoon[413]: IKE Packet: receive success. (Responder, Main-Mode message 1).

May 18 18:58:13 mms.private racoon[413]: >>>>> phase change status = Phase 1 started by us

May 18 18:58:13 mms.private racoon[413]: IKE Packet: transmit success. (Responder, Main-Mode message 2).

May 18 18:58:13 mms.private racoon[413]: IKE Packet: receive success. (Responder, Main-Mode message 3).

May 18 18:58:13 mms.private racoon[413]: IKE Packet: transmit success. (Responder, Main-Mode message 4).

May 18 18:58:13 mms.private racoon[413]: IKEv1 Phase 1 AUTH: success. (Responder, Main-Mode Message 5).

May 18 18:58:13 mms.private racoon[413]: IKE Packet: receive success. (Responder, Main-Mode message 5).

May 18 18:58:13 mms.private racoon[413]: IKEv1 Phase 1 Responder: success. (Responder, Main-Mode).

May 18 18:58:13 mms.private racoon[413]: IKE Packet: transmit success. (Responder, Main-Mode message 6).

May 18 18:58:13 mms.private racoon[413]: IPSec Phase 1 established (Initiated by peer).

May 18 18:58:14 mms.private racoon[413]: IKEv1 Phase 2 Initiator: dropped. (Can't continue Phase 2 without valid Phase 1).

May 18 18:58:14 mms.private racoon[413]: can't start the quick mode, invalid linked ISAKMP-SA

May 18 18:58:16 mms.private racoon[413]: IKEv1 Phase 2 Initiator: dropped. (Can't continue Phase 2 without valid Phase 1).

May 18 18:58:16 mms.private racoon[413]: can't start the quick mode, invalid linked ISAKMP-SA

May 18 18:58:18 mms.private racoon[413]: IKEv1 Phase 2 Initiator: dropped. (Can't continue Phase 2 without valid Phase 1).

May 18 18:58:18 mms.private racoon[413]: can't start the quick mode, invalid linked ISAKMP-SA

May 18 18:58:21 mms.private racoon[413]: IKE Packet: transmit success. (Information message).

May 18 18:58:21 mms.private racoon[413]: IKEv1 Information-Notice: transmit success. (Delete ISAKMP-SA).

May 18 18:58:23 mms.private racoon[413]: IKEv1 Phase 2 Initiator: dropped. (Can't continue Phase 2 without valid Phase 1).

May 18 18:58:23 mms.private racoon[413]: can't start the quick mode, invalid linked ISAKMP-SA

May 18 18:58:31 mms.private racoon[413]: IKEv1 Phase 2 Initiator: dropped. (Can't continue Phase 2 without valid Phase 1).

May 18 18:58:31 mms.private racoon[413]: can't start the quick mode, invalid linked ISAKMP-SA

May 18 18:58:47 mms.private racoon[413]: IKEv1 Phase 2 Initiator: dropped. (Can't continue Phase 2 without valid Phase 1).

May 18 18:58:47 mms.private racoon[413]: can't start the quick mode, invalid linked ISAKMP-SA

May 18 18:59:04 mms.private racoon[413]: IKEv1 Phase 2 Initiator: dropped. (Can't continue Phase 2 without valid Phase 1).

May 18 18:59:04 mms.private racoon[413]: can't start the quick mode, invalid linked ISAKMP-SA

May 18 18:59:18 mms.private racoon[413]: IKE Packet: receive success. (Information message).

Reply

Answer 3

May 18, 2014 4:05 AM in response to burnduck

So ridiculously frustrating - too often when I roll out what should be a fully functional update from Apple recently, it ends up in me looking like a fool in front of clients and having to make excuses for a company that should be providing better. I get that things happen, but the regularity of it recently is pretty unacceptable.

When I can, I'm going to try a backup and restore of the server, which is no small task. Thanks again, Apple.

Reply

Answer 4

May 21, 2014 7:05 AM in response to Chris Billett

I have the same problem - Mini server and incoming VPN connections don't work from iOS devices after applying 10.9.3

Reply

Answer 5

May 21, 2014 7:57 AM in response to Chris Billett

Check under Sharing in System Preferences.

Select File Sharing and click on Options.

Ensure that any listed accounts, under "Windows File Sharing" are ticked on and provide the required passwords.

Retry VPN

Reply

Answer 6

May 21, 2014 8:07 AM in response to dwbrecovery

While I appreciate that might help, and will test if I can set up an alternative environment, what does Windows File Sharing (or, presumably, SMB?) have to do with VPN? I have recently disabled SMB (though longer ago than I applied the update, so VPN still worked for a while afterwards) because of the problems it causes with shared Microsoft Word documents (or any files that use variations on safe save document management) and it isn't an option to turn it on because clients seem to randomly flick back to SMB over AFP no matter how I create the original shortcuts... which is just another annoying bug. Cheers, Apple!

Useful response under other circumstances though, so thanks - I just can't be forced to decide over working Microsoft Office document sharing or VPN!

Reply

Answer 7

May 21, 2014 8:25 AM in response to Chris Billett

My understanding of this is that these accounts / passwords are used by pppd during the establishment of VPN. I had this 'Authentication Failed' issue back with Mavs 10.9.1.

I checked, I do not have SMB File Sharing enabled but VPN does work.

I use network accounts now for VPN and found that there were no entries for these accounts under "Windows File Sharing".

Reply

Answer 8

May 21, 2014 8:33 AM in response to dwbrecovery

Very interesting... I have just enabled it on only my own account as I can be sure to not connect to the share point, and it now works on L2TP, though still comes up with Authentication Failed for PPTP (which I don't configure for my users anyway, so that's fine). Perhaps I can block SMB at a share level and leave these accounts on for users who need VPN access, though it's a really messy way of doing it. All very useful for when the Server support team call me in a couple of hours though, as I can run it past them and see if there's a more complete way of solving without having to re-enable services I would rather have dormant.

Anyway, thanks!

Reply

Answer 9

May 21, 2014 9:56 AM in response to Chris Billett

Doh!

I'm not exactly what was going on with my configuration but the dynamic DNS service I use wasn't current. I figured this out after I was able to establish a VPN connection over my local network but not externally. Updated the DNS and no problem.

But earlier in the day I wasn't able to connect over the local net either and I have no idea what I did for that problem to go away. I already had Windows file sharing enabled. I started/stopped the VPN service, repaired permissions, and added, then dropped PPTP.

Anyway, the VPN service is working again although the true test will come when my daughter tries to connect from Singapore.

Reply

Answer 10

May 21, 2014 12:49 PM in response to dwbrecovery

Good to know...thanks for sharing; however, the SMB file sharing...enable it on the server or the client...both?

Reply

Answer 11

May 21, 2014 3:52 PM in response to Ralston Champagnie

Check on the Server with VPN service enabled.

Check under Sharing in System Preferences.

Select File Sharing and click on Options. File Sharing does not need to be enabled.

Ensure that any listed accounts, under "Windows File Sharing" are ticked on and provide the required passwords.

The SMB File Sharing does not need to be ticked.

retry VPN

Reply

Answer 12

May 24, 2014 8:38 AM in response to burnduck

Error 789 is - The L2TP connection attempt failed because the security layer encountered a processing error during initial negotiations with the remote computer. refer: http://blogs.technet.com/b/rrasblog/archive/2009/08/12/troubleshooting-common-vp n-related-errors.aspx

Managed to get a win 8.1 machine ( VM ) to connect via VPN to Mavs 10.9.3 with Server 3.1.2.

Under Properties -> Security Tab, settings are:

Type of VPN: L2TP/IPsec

Advanced settings -> enter the shared secret

Data encryption: Maximum strength

Authentication -> Allow protocol MS-CHAP V2 only, no other protocols selected

Give it a try

Reply