Trouble Getting My VPN Server To Work

Question

Level 1

9 points

Trouble Getting My VPN Server To Work

So when I try to connect my clients to my VPN server within my network using the server's local IP everything works fine. However, once I try connecting my laptop to the VPN through my public IP it doesn't work. My Windows 10 laptop says "The L2TP connection attempt failed because the security layer encountered a processing error during initial negotiations with the remote computer." I have double checked and my username, password, and secret which are all entered correctly. Also I have double checked and I know that my router is port forwarded correctly. Any solutions would be greatly appreciated!

Here is the server side VPN log:

2018-09-11 21:54:59 EDT Incoming call... Address given to client = 10.0.0.4

Tue Sep 11 21:54:59 2018 : Directory Services Authentication plugin initialized

Tue Sep 11 21:54:59 2018 : Directory Services Authorization plugin initialized

Tue Sep 11 21:54:59 2018 : publish_entry SCDSet() failed: Success!

Tue Sep 11 21:54:59 2018 : L2TP incoming call in progress from '192.168.1.254'...

Tue Sep 11 21:54:59 2018 : L2TP received SCCRQ

Tue Sep 11 21:54:59 2018 : L2TP sent SCCRP

2018-09-11 21:55:03 EDT Incoming call... Address given to client = 10.0.0.5

Tue Sep 11 21:55:03 2018 : Directory Services Authentication plugin initialized

Tue Sep 11 21:55:03 2018 : Directory Services Authorization plugin initialized

Tue Sep 11 21:55:03 2018 : publish_entry SCDSet() failed: Success!

Tue Sep 11 21:55:03 2018 : L2TP incoming call in progress from '192.168.1.254'...

Tue Sep 11 21:55:03 2018 : L2TP received SCCRQ

Tue Sep 11 21:55:03 2018 : L2TP sent SCCRP

2018-09-11 21:55:16 EDT --> Client with address = 10.0.0.2 has hungup

2018-09-11 21:55:17 EDT --> Client with address = 10.0.0.3 has hungup

2018-09-11 21:55:19 EDT --> Client with address = 10.0.0.4 has hungup

2018-09-11 21:55:23 EDT --> Client with address = 10.0.0.5 has hungup

IMAC (RETINA 5K, 27-INCH, LATE 2015)

Posted on Sep 11, 2018 6:58 PM

Reply

Answer 1

MrHoffman

Level 10

146,197 points

Oct 13, 2018 10:28 AM in response to ranman9086

Avoid 192.168.0.0/24 and 192.168.1.0/24, as VPNs are based on IP routing and IP routing isn't fond of having the same subnet on both ends of a link, and those subnets are used all over the place. Use a different subnet somewhere else in 192.168.0.0/16, or in 172.16.0.0/12, or in 10.0.0.0/8. IP routing errors will routinely derail VPN connections.

Also consider acquiring a firewall with an embedded VPN server. Mixing VPNs and NAT and a host-based VPN server has long been "fun", and a firewall-embedded VPN server is in a much better network position. That also works when the host is down.

The VPN server embedded in macOS is also soon to be discontinued, too.

Reply

Answer 2

Oct 13, 2018 10:28 AM in response to ranman9086

I have exactly the same issue today. I updated OSX Server yesterday and today VPN does not work. I've attempted to reconfigure but no joy. I re-configured as per this post https://blog.macstadium.com/blog/setup-a-vpn-server-with-macos-sierra-server-10- 12.

I haven't used the VPN for a couple of months but it was working previously. It's possible the servers OS has gone from Sierra to High Sierra since I last used it hence attempting to re-configuring using the High Sierra specific example.

Did you get yours resolved?

...

2018-09-22 12:03:45 BST

Incoming call... Address given to client = 10.0.0.155

Sat Sep 22 12:03:45 2018 : Directory Services Authentication plugin initialized

Sat Sep 22 12:03:45 2018 : Directory Services Authorization plugin initialized

Sat Sep 22 12:03:45 2018 : publish_entry SCDSet() failed: Success!

Sat Sep 22 12:03:45 2018 : L2TP incoming call in progress from '85.255.234.251'...

Sat Sep 22 12:03:45 2018 : L2TP received SCCRQ

Sat Sep 22 12:03:45 2018 : L2TP sent SCCRP

2018-09-22 12:03:49 BST

Incoming call... Address given to client = 10.0.0.156

Sat Sep 22 12:03:49 2018 : Directory Services Authentication plugin initialized

Sat Sep 22 12:03:49 2018 : Directory Services Authorization plugin initialized

Sat Sep 22 12:03:49 2018 : publish_entry SCDSet() failed: Success!

Sat Sep 22 12:03:49 2018 : L2TP incoming call in progress from '85.255.234.251'...

Sat Sep 22 12:03:49 2018 : L2TP received SCCRQ

Sat Sep 22 12:03:49 2018 : L2TP sent SCCRP

2018-09-22 12:03:50 BST	--> Client with address = 10.0.0.150 has hungup
2018-09-22 12:03:51 BST	--> Client with address = 10.0.0.151 has hungup
2018-09-22 12:03:53 BST	--> Client with address = 10.0.0.152 has hungup
2018-09-22 12:03:57 BST	--> Client with address = 10.0.0.153 has hungup
2018-09-22 12:04:01 BST	--> Client with address = 10.0.0.154 has hungup
2018-09-22 12:04:05 BST	--> Client with address = 10.0.0.155 has hungup
2018-09-22 12:04:09 BST	--> Client with address = 10.0.0.156 has hungup

Reply

Answer 3

MrHoffman

Level 10

146,197 points

Oct 13, 2018 12:16 PM in response to prbarnard

prbarnard : The embedded VPN server and many other services that macOS with Server.app had provided has been deprecated and removed.

For most of us, macOS as a server is dead, and due to be replaced. Or we’re going to be doing a whole lot of piecemeal installation, management, and maintenance of the network services we need on macOS.

Here? Replace the VPN server.

Preferably replace it with a firewall-based embedded VPN server, when operating with a private, NAT’d network. That configuration avoids trying to pass a VPN through NAT, as VPNs and NAT fundamentally operate at cross purposes in a network.

It‘s also possible to migrate to an add-on, host-based VPN server, too.

Reply

Trouble Getting My VPN Server To Work

Similar questions