afctl (Adaptive Firewall) error in 10.8.2

Hi Rusty,

Although I can't help out, I'm feeling some comfort knowing I'm not the only one having this issue; it would seem that Apple's afctl implementation is severly broken at this point. I'm also running 10.8.2, but Server.app is at 2.2.1.

Unfortunately, the emond system is far outside of my knowledge base, so I'm just feeling around in the dark at this point. One strange thing I noticed was that there are 3 identical array values for additionalRulesPaths declared in /etc/emond.d/emond.plist, all of which point to the ServerRoot inside ServerApp. Is this redundancy necessary? Could it cause issues?

I'll be monitoring this thread, so hopefully someone will find a solution.

P.S.: Is that your writeup for configuring the VPN over on the macminicolo.net blog? Lots of good info there, thank you!

Hi Rusty,

Thanks for the quick reply. I can indeed verify that the the "adaptive" part of the Adaptive Firewall is not working on my server, as it's the reason I started digging into this in the first place. Just yesterday I noticed in my logs that a bot was trying to brute-force login as user "admin" (good thing I don't have one!):

-----
2/4/13 8:28:02.864 PM log[241]: auth: Error: od[getpwnam_ext](admin,<NAUGHTY-IP>): No record for user
2/4/13 8:28:02.864 PM log[241]: auth: Error: od(admin,<NAUGHTY-IP>): verify plain: lookup failed for user: admin
2/4/13 8:28:02.865 PM emond[204]: Host at <NAUGHTY-IP> will be blocked for at least 15 minutes
2/4/13 8:28:06.987 PM log[241]: auth: Error: od[getpwnam_ext](admin,<NAUGHTY-IP>): No record for user
2/4/13 8:28:06.987 PM log[241]: auth: Error: od(admin,<NAUGHTY-IP>): verify plain: lookup failed for user: admin
2/4/13 8:28:06.988 PM emond[204]: Host at <NAUGHTY-IP> will be blocked for at least 15 minutes
2/4/13 8:28:07.052 PM emond[204]: Host at <NAUGHTY-IP> will be blocked for at least 15 minutes
2/4/13 8:28:11.097 PM log[241]: auth: Error: od[getpwnam_ext](admin,<NAUGHTY-IP>): No record for user
2/4/13 8:28:11.097 PM log[241]: auth: Error: od(admin,<NAUGHTY-IP>): verify plain: lookup failed for user: admin
2/4/13 8:28:11.098 PM emond[204]: Host at <NAUGHTY-IP> will be blocked for at least 15 minutes
2/4/13 8:28:11.157 PM emond[204]: Host at <NAUGHTY-IP> will be blocked for at least 15 minutes
-----

...and it goes on like that for several hundred lines. I manually added the offending IP to my blacklist and the login attempts stopped immediately. This matches your experience, so at least the behaviors are consistent.

In the logs above, emond appears to be triggering the block appropraitely, but afctl is either ignoring or not receiving the block. As far as I can tell, the problem appears to lie somewhere between emond and afctl, but I'm not sure where or how to troubleshoot that connection.

My server is under AppleCare, so I will give them a call and try to file a bug reportat the very least.

An additional (and confusing) update. The adaptive firewall may actually be doing something on my machine after all, but it's definitely not consistent. While digging through my logs again today, I noticed the events that I've pasted below. Apologies for the wall of text, but I've included the entire transcript of the attack for completeness.

2/6/13 10:48:44.161 PM log[7449]: auth: Error: od[getpwnam_ext](server,<IP-ADDRESS>): No record for user
2/6/13 10:48:44.161 PM log[7449]: auth: Error: od(server,<IP-ADDRESS>): verify plain: lookup failed for user: server
2/6/13 10:48:51.331 PM log[7449]: auth: Error: od[getpwnam_ext](server,<IP-ADDRESS>): No record for user
2/6/13 10:48:51.331 PM log[7449]: auth: Error: od(server,<IP-ADDRESS>): verify plain: lookup failed for user: server
2/6/13 10:48:55.243 PM log[7449]: auth: Error: od[getpwnam_ext](server,<IP-ADDRESS>): No record for user
2/6/13 10:48:55.243 PM log[7449]: auth: Error: od(server,<IP-ADDRESS>): verify plain: lookup failed for user: server
2/6/13 10:49:03.151 PM log[7449]: auth: Error: od[getpwnam_ext](server,<IP-ADDRESS>): No record for user
2/6/13 10:49:03.151 PM log[7449]: auth: Error: od(server,<IP-ADDRESS>): verify plain: lookup failed for user: server
2/6/13 10:49:07.112 PM log[7449]: auth: Error: od[getpwnam_ext](server,<IP-ADDRESS>): No record for user
2/6/13 10:49:07.112 PM log[7449]: auth: Error: od(server,<IP-ADDRESS>): verify plain: lookup failed for user: server
2/6/13 10:49:10.989 PM log[7449]: auth: Error: od[getpwnam_ext](server,<IP-ADDRESS>): No record for user
2/6/13 10:49:10.989 PM log[7449]: auth: Error: od(server,<IP-ADDRESS>): verify plain: lookup failed for user: server
2/6/13 10:49:21.890 PM log[7449]: auth: Error: od[getpwnam_ext](server,<IP-ADDRESS>): No record for user
2/6/13 10:49:21.890 PM log[7449]: auth: Error: od(server,<IP-ADDRESS>): verify plain: lookup failed for user: server
2/6/13 10:49:25.801 PM log[7449]: auth: Error: od[getpwnam_ext](server,<IP-ADDRESS>): No record for user
2/6/13 10:49:25.801 PM log[7449]: auth: Error: od(server,<IP-ADDRESS>): verify plain: lookup failed for user: server
2/6/13 10:49:29.699 PM log[7449]: auth: Error: od[getpwnam_ext](server,<IP-ADDRESS>): No record for user
2/6/13 10:49:29.699 PM log[7449]: auth: Error: od(server,<IP-ADDRESS>): verify plain: lookup failed for user: server
2/6/13 10:49:29.700 PM emond[117]: Host at <IP-ADDRESS> will be blocked for at least 15 minutes
2/6/13 10:49:29.870 PM emond[117]: Host at <IP-ADDRESS> will be blocked for at least 15 minutes
2/6/13 10:49:29.948 PM afctl[25763]: Address already in the blacklist, not added (timeout has been updated)
2/6/13 10:49:29.954 PM emond[117]: 381912569.864889 Host at <IP-ADDRESS> was blocked for 15
2/6/13 10:49:29.954 PM emond[117]: 381912569.864889 Host at <IP-ADDRESS> was blocked for 15
2/6/13 10:49:29.954 PM emond[117]: 381912569.864889 Host at <IP-ADDRESS> was blocked for 15
2/6/13 10:49:33.591 PM log[7449]: auth: Error: od[getpwnam_ext](server,<IP-ADDRESS>): No record for user
2/6/13 10:49:33.591 PM log[7449]: auth: Error: od(server,<IP-ADDRESS>): verify plain: lookup failed for user: server
2/6/13 10:49:33.592 PM emond[117]: Host at <IP-ADDRESS> will be blocked for at least 15 minutes
2/6/13 10:49:33.669 PM afctl[25764]: Address already in the blacklist, not added (timeout has been updated)
2/6/13 10:49:33.675 PM emond[117]: Host at <IP-ADDRESS> will be blocked for at least 15 minutes
2/6/13 10:49:33.754 PM afctl[25765]: Address already in the blacklist, not added (timeout has been updated)
2/6/13 10:49:33.759 PM emond[117]: Host at <IP-ADDRESS> will be blocked for at least 15 minutes
2/6/13 10:49:33.836 PM afctl[25766]: Address already in the blacklist, not added (timeout has been updated)
2/6/13 10:49:37.477 PM log[7449]: auth: Error: od[getpwnam_ext](server,<IP-ADDRESS>): No record for user
2/6/13 10:49:37.477 PM log[7449]: auth: Error: od(server,<IP-ADDRESS>): verify plain: lookup failed for user: server
2/6/13 10:49:37.478 PM emond[117]: Host at <IP-ADDRESS> will be blocked for at least 15 minutes
2/6/13 10:49:37.552 PM afctl[25768]: Address already in the blacklist, not added (timeout has been updated)
2/6/13 10:49:37.558 PM emond[117]: Host at <IP-ADDRESS> will be blocked for at least 15 minutes
2/6/13 10:49:37.633 PM afctl[25769]: Address already in the blacklist, not added (timeout has been updated)
2/6/13 10:49:37.638 PM emond[117]: Host at <IP-ADDRESS> will be blocked for at least 15 minutes
2/6/13 10:49:37.720 PM afctl[25770]: Address already in the blacklist, not added (timeout has been updated)
2/6/13 10:49:41.433 PM log[7449]: auth: Error: od[getpwnam_ext](server,<IP-ADDRESS>): No record for user
2/6/13 10:49:41.433 PM log[7449]: auth: Error: od(server,<IP-ADDRESS>): verify plain: lookup failed for user: server
2/6/13 10:49:41.434 PM emond[117]: Host at <IP-ADDRESS> will be blocked for at least 15 minutes
2/6/13 10:49:41.511 PM afctl[25771]: Address already in the blacklist, not added (timeout has been updated)
2/6/13 10:49:41.516 PM emond[117]: Host at <IP-ADDRESS> will be blocked for at least 15 minutes
2/6/13 10:49:41.597 PM afctl[25772]: Address already in the blacklist, not added (timeout has been updated)
2/6/13 10:49:41.602 PM emond[117]: Host at <IP-ADDRESS> will be blocked for at least 15 minutes
2/6/13 10:49:41.678 PM afctl[25773]: Address already in the blacklist, not added (timeout has been updated)

Note that I've made no changes to this server since my last post, but afctl does indeed appear to be working in those logs; there are no more login attempts from that IP after this excerpt. Additionally, I thought this might be a good sign for progress on getting Apple's KB Article (http://support.apple.com/kb/HT5519) to work, but I still receive the <Notice>: Cannot update the Event Monitor config error.

Also, what's up with the auth errors logging after the IP's already been blocked? And there's a 4-second window preceeding them each time.

Curiouser and couriser.

Thanks for sharing.

I'm in the same boat... ML 10.8.2 with server 2.2.1. I'm having ssh brute force attempts. I've tried to install denyhosts (which doesn't work in ML since TCL wrappers are missing) and am now trying to get sshguard working from macports.

sshguard seems to be doing something, adding stuff to pf firewall, but attack continues as if nothing happens. So I'm looking to make sure that the firewall is up and running as planned. When following the Apple instructions (http://support.apple.com/kb/HT5519), I get the following:

$ sudo /Applications/Server.app/Contents/ServerRoot/usr/libexec/afctl -f

afctl[4803] <Notice>: Cannot update the Event Monitor config

@catkfr

The AF simply creates rules which PF (packet filter) uses.

Did you enable PF? You can find several articles on the topic.

If you want to know if its all working, try consecutive bad logins and see what happens..

Worst case, you get disconnected for xx (15 by default) minutes.

I changed these lines to /Applications/Server.app/Contents/ServerRoot/System/Library/LaunchDaemons/com.a pple.afctl.plist to block IPs after 10 failed login attempts:

<key>ProgramArguments</key>

<array>

<string>/Applications/Server.app/Contents/ServerRoot/usr/libexec/afctl</string>

<string>-T</string>

<string>10</string>

</array>

Therefore the Application Firewall doesn't work correctly, blocked IPs are never removed. I sent a bugreport to Apple, and now I have a "solution". But, yes, the Application Firewall is very buggy and unusable, at least for ssh. I'm using sshguard from Homebrew.

The bugreports 12117251 (Dictionary attacks) and this one are open yet.

Here a some notes:

To get afctl logging work I changed /Applications/Server.app/Contents/ServerRoot/System/Library/LaunchDaemons/com.a pple.afctl.plist like:

<key>ProgramArguments</key>

<array>

<string>/Applications/Server.app/Contents/ServerRoot/usr/libexec/afctl</string>

<string>-v</string>

<string>1</string>

</array>

I added in /etc/asl.plist:

? [= Facility SBS_Security] file /var/log/system.log

and HUPed syslog.

To block the threshold of failed login attempts change in /Applications/Server.app/Contents/ServerRoot/private/etc/emond.d/rules/Adaptive Firewall.plist:

<key>hostBlockThreshold</key>

<integer>10</integer>

Restart emond with

launchctl unload -w /System//Library/LaunchDaemons/com.apple.emond.plist

rm /etc/emond.d/state/state

touch /etc/emond.d/state/state

launchctl load -w /System//Library/LaunchDaemons/com.apple.emond.plist

Restart afctl with

serverctl disable service=com.apple.afctl

killall com.apple.serverd

serverctl enable service=com.apple.afctl

The commands

rm /etc/emond.d/state/state

touch /etc/emond.d/state/state

clear the state of emond, they shouldn't be necessary always.

I wrote:

I added in /etc/asl.plist:

? [= Facility SBS_Security] file /var/log/system.log

and HUPed syslog

This is false! My error. Sorry.