13 Replies Latest reply: Nov 7, 2012 6:18 AM by mmurray47
Frando Level 1 Level 1 (145 points)

Hi,

 

My server is often needing to be hard-reset as it stops responding.  A lot of the services appear to respond, albiet slowly, except for network account logins (AFP not mounitng path).  Reset resolves but seems a little harsh.

 

I have noticed in the logs:

 

11/10/2011 10:29:24.465 servermgrd: servermgr_ipfilter:ipfw config:Notice:Flushed IPv4 rules
11/10/2011 10:29:24.627 servermgrd: servermgr_ipfilter:ipfw config:Notice:Flushed IPv6 rules

 

Any idea if this is indicative of the problem, or nothing to worry about?

 

Thanks.


Mac Pro, Mac OS X (10.7.1)
  • 1. Re: servermgrd ipfw flushed rules
    Ian L-F Level 1 Level 1 (10 points)

    I get this quite often actually.  Any resolution on your end?

  • 2. Re: servermgrd ipfw flushed rules
    Frando Level 1 Level 1 (145 points)

    Nope.  Still getting it.

  • 3. Re: servermgrd ipfw flushed rules
    Chris Connors Level 1 Level 1 (120 points)

    my instance is getting the same console entry, and is accompanied by high (80-100%) CPU utilization by servermgrd. The really odd thing is that Firewall is not even enabled as a service on this instance and never has been.

     

    I am going to try and enable Firewall and alter its settings but this is really strange.

     

    edit:

     

    doesn't seem to have helped:

     

    Feb  4 19:28:05 servermgrd[21053]: servermgr_ipfilter:ipfw config:Notice:Enabled firewall

    Feb  4 19:29:55 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv4 rules

    Feb  4 19:29:56 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv6 rules

    Feb  4 19:29:56 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv4 rules

    Feb  4 19:29:56 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv6 rules

    Feb  4 19:29:56 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv4 rules

    Feb  4 19:29:56 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv6 rules

    Feb  4 19:29:57 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv4 rules

    Feb  4 19:29:57 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv6 rules

    Feb  4 19:29:57 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv4 rules

    Feb  4 19:29:57 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv6 rules

    Feb  4 19:29:57 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv4 rules

    Feb  4 19:29:57 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv6 rules

    Feb  4 19:29:57 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv4 rules

    Feb  4 19:29:57 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv6 rules

    Feb  4 19:29:58 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv4 rules

    Feb  4 19:29:58 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv6 rules

    Feb  4 19:29:58 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv4 rules

    Feb  4 19:29:58 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv6 rules

    Feb  4 19:29:58 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv4 rules

    Feb  4 19:29:58 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv6 rules

    Feb  4 19:30:56 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv4 rules

    Feb  4 19:30:56 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv6 rules

    Feb  4 19:30:58 servermgrd[22068]: servermgr_ipfilter:ipfw config:Notice:Disabled firewall

    Feb  4 19:31:58 servermgrd[22539]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv4 rules

    Feb  4 19:31:59 servermgrd[22539]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv6 rules

    Feb  4 19:32:59 servermgrd[22539]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv4 rules

    Feb  4 19:32:59 servermgrd[22539]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv6 rules

     

    Message was edited by: Chris Connors

  • 4. Re: servermgrd ipfw flushed rules
    RoseValley Level 1 Level 1 (10 points)

    I am getting the same basic errors. I am running an Xserve with Lion 10.7.2 and 5 Xserve RAID arrays, no additional servicies other then VPN and DNS. Here are my error lines.

     

    Feb  9 12:26:55 xsancontroller servermgrd[46915]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv4 rules

    Feb  9 12:26:55 xsancontroller servermgrd[46915]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv6 rules

    Feb  9 14:26:54 xsancontroller servermgrd[47369]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv4 rules

    Feb  9 14:26:54 xsancontroller servermgrd[47369]: servermgr_ipfilter:ipfw config:Notice:Flushed IPv6 rules

     

    Anyone have an idea what this is caused by and how to fix it? PLEASE?!?

     

    Thanks,

    Kevin

  • 5. Re: servermgrd ipfw flushed rules
    Chris Connors Level 1 Level 1 (120 points)

    I think I might know what was causing this. Was anyone else also using the old Server Status Dashboard widget?

  • 6. Re: servermgrd ipfw flushed rules
    Ian L-F Level 1 Level 1 (10 points)

    No.

  • 7. Re: servermgrd ipfw flushed rules
    Chris Connors Level 1 Level 1 (120 points)

    So I am seeing two things.

     

    One is ipfw flushing values, um, agressively; the other is servermgrd consuming nearly all of the cpu resources. I thought they were one problem, but now I am not so sure.

     

    I noticed that any of the Server Tools (Server.app, Server Admin.app) cause spikes in servermgrd activity on the host, but once the values and server state are reported to the respective apps, the servermgrd cpu usage returns to about nil.

     

    I also remembered that the Server Status Dashboard widget was deprecated a few versions ago (10.6?) but like anyone with time machine, why woud I let that stand in the way of me using it.

     

    Last night i did a little experiment. When I invoked Dashboard on a remote laptop, sure enough, the host's servermgrd cpu spiked to ~98% and stayed that way until the process was killed or I removed the Server Status widget from dashboard. I think the widget provokes high (and persistent, since it is continually interrogating the servermgrd even when Dashboard is in the BG) cpu load activity by servermgrd, which may be why it was dropped.

     

    Since I've removed Server Status Widget, I haven't seen any of these servermgrd deadlocks.

     

    Interestingly enough, when servermgrd is in the deadlocked state, is when it seems to report a flurry of ipfw flushes in the console.

     

    My initial concern was that I didn't even use the firewall on this box, why were values being flushed at all?

     

    So I tried a couple of mitigation strategies. Since I'm not using the firewall, i wanted to get it into a 'default state' first, I mv'd /etc/ipfilter/ip_address_groups.plist to /etc/ipfilter/ip_address_groups.plist.bak. then I cp'd /etc/ipfilter/ip_address_groups.default to /etc/ipfilter/ip_address_groups.plist I did the same for /etc/ipfilter/ipfw.conf.

     

    Then, in Server Admin, I enabled the firewall service, turned it on, turned it off, and then disabled it.

     

    For good measure I checked in the Security Preferences panel for the Firewall settings. It was marked 'Off' but just to be certain, I turned it on, then turned it off.

     

    But I was still seeing servermgrd deadlocks accompanied by flurries of ipfw flushes. Until last night, when I removed Server Status dashboard widget.

     

    Since then I've still seen some ipfw flushes, but they come in pairs, 2 hours apart, not by the dozens, seconds apart. I have to believe they are related. They are also not accompanied by high CPU utilization at all.

     

    So fingers crossed, I think this may be the resolution.

  • 8. Re: servermgrd ipfw flushed rules
    Ian L-F Level 1 Level 1 (10 points)

    Not for those of us who weren't running that widget to begin with.

  • 9. Re: servermgrd ipfw flushed rules
    Chris Connors Level 1 Level 1 (120 points)

    Fair enough. What have you tried to troubleshoot the situation?

  • 10. Re: servermgrd ipfw flushed rules
    Mellified Man Level 1 Level 1 (0 points)

    Running the dashboard widget seemed to be the issue for me as well. Killing it off on two machines that were running it has resulted in normal behavior again.

  • 11. Re: servermgrd ipfw flushed rules
    Kaleberg Level 1 Level 1 (5 points)

    I've run into this problem with 10.7.3. servermgrd chomps up to 99% of a CPU whenever I use Server to look at the resource usage history. If I look at some other item (e.g. Sharing or Address Book status), servermgrd calms down. That's my fix for now, but I've started poking around with fs_usage to see if I can figure out what is going on.

  • 12. Re: servermgrd ipfw flushed rules
    cj00 Level 1 Level 1 (10 points)

    Event with 10.7.4 and without a widget, servermgrd loves to deadlock:

     

    Jun 24 18:19:56 mac01 servermgrd[437]: [437] error in getAndLockContext: flock(servermgr_info) FATAL time out
    Jun 24 18:19:56 mac01 servermgrd[437]: [437] process will force-quit to avoid deadlock
    Jun 24 18:19:56 mac01 servermgrd[437]: outstanding requests are: (
                {
                Command = getHistory;
                Module = "servermgr_info";
                Timestamp = "2012-06-24 16:17:51 +0000";
            },
                {
                Command = getState;
                Module = "servermgr_info";
                Timestamp = "2012-06-24 16:17:51 +0000";
            },
                {
                Command = Idle;
                Module = "servermgr_info";
                Timestamp = "2012-06-24 16:18:50 +0000";
            }
        )
    Jun 24 18:19:56 mac01 com.apple.launchd[1] (com.apple.servermgrd[437]): Exited with code: 1

  • 13. Re: servermgrd ipfw flushed rules
    mmurray47 Level 1 Level 1 (0 points)

    Yes!