L2TP VPN not working; PPTP working

Question

Level 1

0 points

L2TP VPN not working; PPTP working

I have an Intel Xserve 10.4.10 running as a gateway server and doing quite well in the process. My one issue is not getting the L2TP to work properly. PPTP works fine.

When my Apple clients try to connect, I see the following in the VPN Log:
*Note (192.168.1.10-14) have been reserved for L2TP VPN clients.

Tue Sep 25 09:34:44 2007 : Directory Services Authentication plugin initialized
Tue Sep 25 09:34:44 2007 : Directory Services Authorization plugin initialized
Tue Sep 25 09:34:44 2007 : L2TP incoming call in progress
Tue Sep 25 09:34:44 2007 : L2TP received SCCRQ
Tue Sep 25 09:34:44 2007 : L2TP sent SCCRP
Tue Sep 25 09:43:19 2007 : L2TP received AVP with bad length... AVP type = 0
2007-09-25 09:43:19 PDT --> Client with address = 192.168.1.10 has hungup
Tue Sep 25 09:43:20 2007 : L2TP received AVP with bad length... AVP type = 0
2007-09-25 09:43:20 PDT --> Client with address = 192.168.1.11 has hungup
Tue Sep 25 09:43:21 2007 : L2TP received AVP with bad length... AVP type = 0
2007-09-25 09:43:21 PDT --> Client with address = 192.168.1.12 has hungup
Tue Sep 25 09:43:22 2007 : L2TP received AVP with bad length... AVP type = 0
2007-09-25 09:43:22 PDT --> Client with address = 192.168.1.13 has hungup
Tue Sep 25 09:43:23 2007 : L2TP received AVP with bad length... AVP type = 0
2007-09-25 09:43:23 PDT --> Client with address = 192.168.1.14 has hungup

I'm assuming my problem lies with the 'AVP with bad length' error message. But I haven't had any luck researching what exactly this means per the Apple OS.

I have the following ports opened to accept UDP: 500, 1701, 1723 (for PPTP), and 4500.

The server and clients are on different IP ranges, etc. Please someone help!

Thanks.

Mac OS X (10.4.10)

Posted on Sep 25, 2007 11:16 AM

Reply

Answer 1

MacLemon

Level 2

455 points

Sep 25, 2007 12:47 PM in response to BMolls

Don't forget to allow ESP (Protocol 50) on your firewall as well! (Common oversight!)
MacLemon

Reply

Answer 2

BMolls Author

Level 1

0 points

Sep 25, 2007 3:06 PM in response to MacLemon

I just tried it again after opening ESP Port 50, but same results. Here's the full error log. It almost seems as if the LT2P part of the VPN server is broken because it keeps looping to new IP addresses.

Sorry for the long log in advance...

----

2007-09-25 14:59:28 PDT Incoming call... Address given to client = 192.168.1.10
Tue Sep 25 14:59:28 2007 : Directory Services Authentication plugin initialized
Tue Sep 25 14:59:28 2007 : Directory Services Authorization plugin initialized
Tue Sep 25 14:59:28 2007 : L2TP incoming call in progress
Tue Sep 25 14:59:28 2007 : L2TP received SCCRQ
Tue Sep 25 14:59:28 2007 : L2TP sent SCCRP
2007-09-25 14:59:29 PDT Incoming call... Address given to client = 192.168.1.11
Tue Sep 25 14:59:29 2007 : Directory Services Authentication plugin initialized
Tue Sep 25 14:59:29 2007 : Directory Services Authorization plugin initialized
Tue Sep 25 14:59:29 2007 : L2TP incoming call in progress
Tue Sep 25 14:59:29 2007 : L2TP received SCCRQ
Tue Sep 25 14:59:29 2007 : L2TP sent SCCRP
2007-09-25 14:59:30 PDT Incoming call... Address given to client = 192.168.1.12
Tue Sep 25 14:59:30 2007 : Directory Services Authentication plugin initialized
Tue Sep 25 14:59:30 2007 : Directory Services Authorization plugin initialized
Tue Sep 25 14:59:30 2007 : L2TP incoming call in progress
Tue Sep 25 14:59:30 2007 : L2TP received SCCRQ
Tue Sep 25 14:59:30 2007 : L2TP sent SCCRP
2007-09-25 14:59:31 PDT Incoming call... Address given to client = 192.168.1.13
Tue Sep 25 14:59:31 2007 : Directory Services Authentication plugin initialized
Tue Sep 25 14:59:31 2007 : Directory Services Authorization plugin initialized
Tue Sep 25 14:59:31 2007 : L2TP incoming call in progress
Tue Sep 25 14:59:31 2007 : L2TP received SCCRQ
Tue Sep 25 14:59:31 2007 : L2TP sent SCCRP
2007-09-25 14:59:32 PDT Incoming call... Address given to client = 192.168.1.14
Tue Sep 25 14:59:32 2007 : Directory Services Authentication plugin initialized
Tue Sep 25 14:59:32 2007 : Directory Services Authorization plugin initialized
Tue Sep 25 14:59:32 2007 : L2TP incoming call in progress
Tue Sep 25 14:59:32 2007 : L2TP received SCCRQ
Tue Sep 25 14:59:32 2007 : L2TP sent SCCRP
Tue Sep 25 15:00:28 2007 : L2TP received AVP with bad length... AVP type = 0
2007-09-25 15:00:28 PDT --> Client with address = 192.168.1.10 has hungup
Tue Sep 25 15:00:29 2007 : L2TP received AVP with bad length... AVP type = 0
2007-09-25 15:00:29 PDT --> Client with address = 192.168.1.11 has hungup
Tue Sep 25 15:00:30 2007 : L2TP received AVP with bad length... AVP type = 0
2007-09-25 15:00:30 PDT --> Client with address = 192.168.1.12 has hungup
Tue Sep 25 15:00:31 2007 : L2TP received AVP with bad length... AVP type = 0
2007-09-25 15:00:31 PDT --> Client with address = 192.168.1.13 has hungup
Tue Sep 25 15:00:32 2007 : L2TP received AVP with bad length... AVP type = 0
2007-09-25 15:00:32 PDT --> Client with address = 192.168.1.14 has hungup
2007-09-25 15:01:44 PDT Incoming call... Address given to client = 192.168.1.10
Tue Sep 25 15:01:44 2007 : Directory Services Authentication plugin initialized
Tue Sep 25 15:01:44 2007 : Directory Services Authorization plugin initialized
Tue Sep 25 15:01:44 2007 : L2TP incoming call in progress
Tue Sep 25 15:01:44 2007 : L2TP received SCCRQ
Tue Sep 25 15:01:44 2007 : L2TP sent SCCRP
2007-09-25 15:01:45 PDT Incoming call... Address given to client = 192.168.1.11
Tue Sep 25 15:01:45 2007 : Directory Services Authentication plugin initialized
Tue Sep 25 15:01:45 2007 : Directory Services Authorization plugin initialized
Tue Sep 25 15:01:45 2007 : L2TP incoming call in progress
Tue Sep 25 15:01:45 2007 : L2TP received SCCRQ
Tue Sep 25 15:01:45 2007 : L2TP sent SCCRP
2007-09-25 15:01:46 PDT Incoming call... Address given to client = 192.168.1.12
Tue Sep 25 15:01:46 2007 : Directory Services Authentication plugin initialized
Tue Sep 25 15:01:46 2007 : Directory Services Authorization plugin initialized
Tue Sep 25 15:01:46 2007 : L2TP incoming call in progress
Tue Sep 25 15:01:46 2007 : L2TP received SCCRQ
Tue Sep 25 15:01:46 2007 : L2TP sent SCCRP
2007-09-25 15:01:47 PDT Incoming call... Address given to client = 192.168.1.13
Tue Sep 25 15:01:47 2007 : Directory Services Authentication plugin initialized
Tue Sep 25 15:01:47 2007 : Directory Services Authorization plugin initialized
Tue Sep 25 15:01:47 2007 : L2TP incoming call in progress
Tue Sep 25 15:01:47 2007 : L2TP received SCCRQ
Tue Sep 25 15:01:47 2007 : L2TP sent SCCRP
2007-09-25 15:01:48 PDT Incoming call... Address given to client = 192.168.1.14
Tue Sep 25 15:01:48 2007 : Directory Services Authentication plugin initialized
Tue Sep 25 15:01:48 2007 : Directory Services Authorization plugin initialized
Tue Sep 25 15:01:48 2007 : L2TP incoming call in progress
Tue Sep 25 15:01:48 2007 : L2TP received SCCRQ
Tue Sep 25 15:01:48 2007 : L2TP sent SCCRP
Tue Sep 25 15:02:43 2007 : L2TP received AVP with bad length... AVP type = 0
2007-09-25 15:02:43 PDT --> Client with address = 192.168.1.10 has hungup
Tue Sep 25 15:02:44 2007 : L2TP received AVP with bad length... AVP type = 0
2007-09-25 15:02:44 PDT --> Client with address = 192.168.1.11 has hungup
Tue Sep 25 15:02:45 2007 : L2TP received AVP with bad length... AVP type = 0
2007-09-25 15:02:45 PDT --> Client with address = 192.168.1.12 has hungup
Tue Sep 25 15:02:46 2007 : L2TP received AVP with bad length... AVP type = 0
2007-09-25 15:02:46 PDT --> Client with address = 192.168.1.13 has hungup
Tue Sep 25 15:02:47 2007 : L2TP received AVP with bad length... AVP type = 0
2007-09-25 15:02:47 PDT --> Client with address = 192.168.1.14 has hungup

Reply

Answer 3

MacLemon

Level 2

455 points

Sep 25, 2007 3:18 PM in response to BMolls

ESP is not port 50, but Protocol 50, it is on the same layer as TCP, UDP, ICMP etc. It is not a TCP port!

What kind of client is trying to connect to your Mac OS X Server? How many network interfaces does your server have? Which one is the primary interface? On which interface is your client trying to connect to?
MacLemon

Reply

Answer 4

BMolls Author

Level 1

0 points

Sep 25, 2007 3:28 PM in response to MacLemon

The client is a MacBook Pro running 10.4.10 using an Aircard with IP address of 66.x.x.x.

OS X Server is running as a Gateway Router, with en0 as the interface to the outside and en1-3 (bond0)utilizing Link Aggregation internally.

The client is trying to connect with bond0.

I'm stumped at why PPPTP works but not L2TP at this point. I'm using NATural for port forwarding for many different services within the company and everything is working great.

How do I enable Protocol 50 on the XServe?

Reply

Answer 5

BMolls Author

Level 1

0 points

Sep 25, 2007 4:04 PM in response to BMolls

I just checked the Services list under Firewall and 'ESP - Encapsulating Security Payload protocol' is enabled.

Reply

Answer 6

MacLemon

Level 2

455 points

Sep 26, 2007 12:21 AM in response to BMolls

OS X Server is running as a Gateway Router, with en0 as the interface to the outside and en1-3 (bond0)utilizing Link Aggregation internally.

The client is trying to connect with bond0.

So you are trying to connect to the internal interface of your Server from the outside? This will definitely not work.
I am assuming that your en0 is the primary interface of your server (since you did not explicitly provide this info) with your external IP. You need to connect to that external IP on your primary interface on the server. L2TP will always try to answer from the primary interface of the server.

I'm stumped at why PPPTP works but not L2TP at this point. I'm using NATural for port forwarding for many different services within the company and everything is working great.

PPTP connects similar to any other service with no server initiated response, whereas L2TP actively connects back to the VPN client which is exactly the problem here. (Plus a small routing problem with L2TP on Mac OS X Server.)
MacLemon

Reply

Answer 7

BMolls Author

Level 1

0 points

Sep 26, 2007 8:47 AM in response to MacLemon

I'm sorry...I'm not doing a very good job describing my setup. Yes, I'm trying to connect to my external/primary interface, en0 on the server. En0 is connected to our main external ip address and that is where the MacBook Pro client is trying to connect using L2TP VPN. En0 is the outside NIC if that makes sense.

What I meant earlier is that within the settings of the VPN server, I have connections for L2TP using the IP range: 192.168.1.10 - .14. So I want to forward the clients to those assigned addresses.

Reply

Answer 8

HarryMcQ

Level 1

0 points

Nov 16, 2007 1:32 AM in response to BMolls

I am having the exact same problem. Log looks exactly the same. Cannot connect to new Xeon Xserve via L2TP, only PPTP. Have had a similar problem with Mac Pro. I haven't had this problem with PPC hardware. Could L2TP be broken on 10.4 Intel? Maybe it is an IPsec problem... Does anyone have any ideas?

Reply

Answer 9

Nov 16, 2007 5:07 AM in response to BMolls

I recognize this as having been a problem for some people earlier too.
On AFP548 had it mentioned 2005 from some guy but no real explanation to the cause.

A 192.168.1.0/24 net (you have this on your LAN?) isn't that good a choice but you seem to know there can't be two networks with the same settings connected to each other.

I guess you meant TCP port 1723 and GRE protocol open for PPTP?

The en1-3 bonded interface bit have me bewildered. I know there are 2 LOM interfaces in Xserve Intel (using the same ports as the regular Interfaces) but I thought you couldn't use those for regular use?

Might be better getting an ethernet card and put it in the machine (if you have room for it) or put a NAT router/firewall inbetween the server and Internet if you want a bonded interface for LAN.

Reply