DanIsaacson

Q: L2TP VPN and Back to My Mac

Can someone please confirm there is a conflict with the OS X Lion Server L2TP VPN and the Back to My Mac service.

 

I'm getting IKE packet errors on our VPN server and suspect someone on the network has Back to My Mac enabled.

 

Feb 14 12:12:12 server racoon[196]: IKE Packet: receive failed. (Responder, Quick-Mode Message 1).

 

"Authentication Failed" from the client side.  But after a few attempts it will connect... strange.

IPv4 IPv6 issues?

 

My router is non-Apple

and has the following NAT entries forwared to the Mac Server for the L2TP VPN

 

UDP 500

UDP 1701

UDP 4500

 

it also has UPNP enabled.

 

Does Back to My Mac use UPNP, IPv4, or IPv6 only?

 

Is Back to My Mac conflicting with the VPN service in Lion Server?

If so, WHY is this not documented?

Mac mini, Mac OS X (10.7.3), Lion Server (10.7.3)

Posted on Feb 13, 2012 5:44 PM

Close

Q: L2TP VPN and Back to My Mac

  • All replies
  • Helpful answers

  • by Chris.Crisp,

    Chris.Crisp Chris.Crisp Feb 17, 2012 11:08 PM in response to DanIsaacson
    Level 1 (0 points)
    Feb 17, 2012 11:08 PM in response to DanIsaacson

    i'm pretty sure it's conflicting with Back to My Mac. i'd been fiddling around with it for days why i couldn't establish a VPN connection on a VPN gateway (win2k8r2) behind Airport Extreme (Back to My Mac) enabled.

     

    i thought at first it was the firewall, but it was not, then i checked the ports, i even tried PPTP, but i couldn't establish a successful connection even though i have the same VPN setup as with the one in our workplace.

     

    i then called Apple Support about this issue, i asked what ports Back to My Mac are using, he told me a dozen of ports which includes the ports for L2TP (1701, 4500, 500). and some in PPTP ports too i think, i can't remember them all.

     

    it would be nice though if the two (VPN and Back to My Mac) can run without conflicts. but yea... i knew already at first when i first used Back to My Mac it's basically a VPN linked to your Apple ID. simple/good implementation by Apple but i wish they used different ports right?

     

    edit: actually it is documented, it is written somewhere in Apple Support that you have to turn off Back to My Mac/MobileMe to use VPN. i just read it the other day.

  • by DanIsaacson,

    DanIsaacson DanIsaacson Feb 22, 2012 4:31 PM in response to Chris.Crisp
    Level 1 (0 points)
    Feb 22, 2012 4:31 PM in response to Chris.Crisp

    So I've found the offical word burried in an Apple Document... 

    http://support.apple.com/kb/TS1629

     

    scroll down to Port 4500 which is used for IPSec VPN and Back to My Mac...

     

    4500UDPIKE NAT Traversal-ipsec-msftMac OS X Server VPN service, Back to My Mac (MobileMe, Mac OS X v10.5 or later).
    Note: VPN and MobileMe are mutually exclusive when configured through an Apple access point (such as an AirPort Base Station); MobileMe will take precedence.


    NOTE TO APPLE:

    1) Do NOT create network services that conflict with well known and used TCP and UDP ports.

    2) If you create two conflicting services. Please MAKE A NOTE IN THE MANUAL for OS X Server so Admins are aware of the problem.

  • by QuickTimeX,

    QuickTimeX QuickTimeX Dec 8, 2012 1:16 PM in response to DanIsaacson
    Level 1 (0 points)
    Dec 8, 2012 1:16 PM in response to DanIsaacson

    This is so disgusting. This issue wasted me hours of time.

    Is it not possible for Apple to pick different ports?

  • by TheMomentum,

    TheMomentum TheMomentum Sep 7, 2015 8:52 AM in response to QuickTimeX
    Level 1 (0 points)
    Sep 7, 2015 8:52 AM in response to QuickTimeX

    This is still prevalent today on El Capitan. If anyone has OS X server installed, do not turn on Back To My Mac if you plan on having a VPN setup.

     

    Doing so knocked my server out, and i had to go through a significant amount of work to fix it.

     

    To fix:

     

    Turn off Back To My Mac, and do a kickstart

     

    sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/k ickstart -activate -configure -access -off -restart -agent -privs -all -allowAccessFor -allUsers              

     

    Then do a

    sudo launchctl stop com.apple.racoon

    sudo launchctl start com.apple.racoon

     

    and reboot, and when the server comes back up restart the VPN service through the OS X server console .

  • by thavanam,

    thavanam thavanam Jan 11, 2016 10:16 PM in response to TheMomentum
    Level 1 (0 points)
    Jan 11, 2016 10:16 PM in response to TheMomentum

    Hi,

     

    I have tried your post, but still no luck.

     

    My VPN status shows Available - Reachability unknown

     

    I am able connect my vpn from another device on the same network. I am not able to connect from internet.

    I am getting the following error when I try to connect from my iPhone 4s.

    The L2TP-VPN server id not respond. Try reconnecting. If the problem continues, verify your settings and contact your administrator.

     

    OS - OS X El Capitan

    version 10.11.2

     

    Please help if any one knows the solution.

  • by JesseDegenerate,

    JesseDegenerate JesseDegenerate Jan 13, 2016 12:03 PM in response to TheMomentum
    Level 1 (0 points)
    Jan 13, 2016 12:03 PM in response to TheMomentum

    wow, i'm on 10.10.5 for my server (10.11.2 for clients and iOS 9.2) and was pretty upset when i read your post, as i had just enabled it on a whim as i do a lot of remote stuff, but i figured i would play with it for some fun.

     

    I was able to get it going again rather quickly with both still active, my server showing up in back to mac, AND connecting via L2TP. There's a lot of information out there saying it's impossible, so i just figured i would say it's apparently not, so don't give up, although it may just have been fixed.

     

    I just re-forwarded the ports, toggled VPN. good to go.

  • by Mrwhoopie1,

    Mrwhoopie1 Mrwhoopie1 Mar 27, 2016 11:33 AM in response to JesseDegenerate
    Level 1 (4 points)
    Mar 27, 2016 11:33 AM in response to JesseDegenerate

    What did you do top make it work?Because i run all the latest software and since yesterday i activate "back to my Mac" and i can't connect via L2TP i'm not @ home right now and i remotely shut down "remote to my Mac" but still no VPN via L2TP

  • by Andreas Amann,

    Andreas Amann Andreas Amann Jun 4, 2016 7:36 PM in response to Mrwhoopie1
    Level 5 (7,845 points)
    Mac OS X
    Jun 4, 2016 7:36 PM in response to Mrwhoopie1

    I finally got things working - in additional to disabling BTMM on your server, you also have to disable it on your Airport base station (in case you use one of those) - I tested things and L2TP now works for me after disabling BTMM on both.