5 Replies Latest reply: Jun 2, 2010 3:42 PM by GeekMan
GeekMan Level 1 Level 1 (10 points)
Hi all.

I'm setting up my second Snow Leopard Server machine and I'm having the most frustrating networking problems.

I'm setting up a server with two NICs, and for some bizarre reason connection will NOT work. Trying to ping within the subnet of the inoperative connection produces "no route to host" errors. Switch the IP configurations between the two NICs, and the problem follows the primary configuration. The same network settings work correctly on a different machine, so the problem is definitely software.

I can't help wondering if this is related to source-based routing in 10.6. But I've tried flushing the routing table to no avail.

This exact same issue happened with the other Snow Leopard server I set up. After a few days of idling, the problems magically resolved themselves. Unfortunately, I don't have time for that to happen with this server.

Any suggestions?

Intel Xeon Xserve, Mac OS X (10.6.3)
  • MrHoffman Level 6 Level 6 (13,020 points)
    The forums are littered with dual-controller subnet routing discussions; it's a very common issue.

    Easiest fix: don't use a Mac as an expensive (and slow) IP router (or NAT box). Use an external firewall.

    If you really want to wade into IP routing (and do you actually have enough network traffic to warrant lighting up two gigabit Ethernet connections right now?) then poke around for the existing discussions of setting up the ordering of the controllers and establishing subnet routing live and after each reboot.

    I'm probably going to end up writing an article and posting it, as this arises way too often, and folks are understandably having issues with locating the problem given that most (understandably) don't know they need to look for subnets and static routes and such.

    Existing threads including [this|http://discussions.apple.com/thread.jspa?messageID=10688941] and [this|http://discussions.apple.com/message.jspa?messageID=5697532] are reasonable starting points for your dual-NIC quest.

    One other related gremlin that crops up here is the subnet addressing assignments; you'll usually want to have the (co-resident) NICs using addresses in separate subnets or routing can get squirrelly.

    Or plug in the external firewall (which has other advantages) and use it as the router and firewall and NAT box for your LAN.
  • GeekMan Level 1 Level 1 (10 points)
    Hi there.

    I'm actually not using the XServe for NAT, we just happen to have dual ISPs in the office and I want to be able to access both networks. The network that isn't working contains what should be the primary IP address of the machine.

    I never had any problems with dual networks on OSX machines until Snow Leopard. All of a sudden it's like pulling teeth.
  • MrHoffman Level 6 Level 6 (13,020 points)
    If you want multiple parallel IP paths and (as is usually combined with that) some sort of connection load balancing and (often added) connection failover, then you'll need a widget (firewall/router, usually) that does BGP or analogous.

    For discussions of your particular set-up, kindly see [this|http://discussions.apple.com/thread.jspa?threadID=1302941] and [this|http://discussions.apple.com/thread.jspa?threadID=424626] and [this|http://discussions.apple.com/thread.jspa?threadID=512477] thread.

    For a simpler and more static configuration, that's usually with static routes.

    I don't know that Apple supports IP routing on the box; that's really what you'd need here to get this fully working without the assistance of an external IP router. (IIRC, there's a knob around that turns that on, but I'm not immediately able to locate documentation.)
  • GeekMan Level 1 Level 1 (10 points)
    I think perhaps I'm not explaining myself clearly.

    Not trying to do any bridging or load balancing, just want to be able to access a local subnet. I'd be happy if I just had to set up an extra static route for that. The problem is more elusive.

    I just reformatted the server to start fresh. I enter the IP information for network 1. Plug in the ethernet cable for network 1. I make sure the network interface is at the top of the list, meaning that this should be the default route. Right?

    I enter the IP info for the second network. Apply. Again, everything is still good.

    However, when I plug the ethernet into the second NIC, suddenly, I can't access anything on the internet through the first network anymore, including the gateway for network 1. This makes no sense to me: I've clearly set NIC/Network 1 as my default connection, why is everything suddenly being routed through NIC/Network 2?
  • GeekMan Level 1 Level 1 (10 points)
    GAH!

    Dragged NIC2 to the top of the list in Network Preferences to make it the default route. Hit apply. Tested. The dragged NIC1 back to the (back the way it was before when it wouldn't work) hit apply. Suddenly it works.

    Don't understand, but after a million reboots, and confusing netstat -r results, works the way its supposed to all of a sudden.