Apple Event: May 7th at 7 am PT

Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

Level 1

41 points

Spamassassin Junkmail and Notjunkmail Learning in Mountain Lion

Like many others <https://discussions.apple.com/thread/4181320?start=0&tstart=0>, I wondered what happened to spamassassin and its Bayesian learning mailboxes "junkmail" and "notjunkmail" in OS X Server. Perhaps Apple had automated this process? There certainly isn't any documentation. I didn't think much of it until spam began falling into my server's INBOX, whereupon looking for a fix I discovered that spamassassin learning capabilities still work in Mountain Lion OS X Server, but they aren't automated in any way.

Here's how to train your server to detect the junk mail you receive, beyond the appropriate spam filter and greylist settings Server.app>Mail.

First, confirm that your serveradmin settings know about the spamassassin Bayesian learning mailboxes "junkmail" and "notjunkmail":

sudo serveradmin settings mail:imap | grep junk

mail:imap:junk_mail_userid = "junkmail"

mail:imap:not_junk_mail_userid = "notjunkmail"

Then use Server.app to add the network accounts "junkmail" and "notjunkmail" and edit access so that these accounts only have access to the Mail service. Add these accounts to Mail so that you (and other trusted users) have access to them.

Then dump all your junkmail from your server accounts and other cloud email accounts into the "junkmail" mailbox. Move any messages that aren't junk but marked as such into the mailbox "notjunkmail".

Finally,

sudo /Applications/Server.app/Contents/ServerRoot/usr/libexec/spamassassin/learn_jun k_mail.sh

This will train spamassassin based on the content of these mailboxes, and hopefully you will begin to see amavisd properly mark junkmail as "** JUNK MAIL **" in the subject line.

Mac mini Server (Mid 2010), OS X Mountain Lion (10.8.2), Lion Server, EyeTV HD, Turbo.264 HD

Posted on Nov 4, 2012 2:40 AM

34 replies

UptimeJeff

Level 4

3,479 points

Dec 5, 2012 7:29 PM in response to essandess

I've never been happy with bayes training since the old cyrus shared mailbox days.

Users shouldn't need to redirect messages and sharing access to junkmail/notjunkmail imap accounts is cumbersome.

Today, I tackled this with a quick script which does this:

- Scans every mail user for 2 folders (junkmail/notjunkmail)

- Feeds those messages to sa-learn, to train the bayes filters.

I will add

- purge the user junkmail/notjunkmail folders of messages > 30 days old.

End result, users simply drag messages within their own imap account.

The script can specifcy the standard 'Junk' folder so that client apps that filter to 'Junk' will contribute to the common bayes DB.

This creates an easy to use training system which allows all users to contribute easily.

Requires no server config, users only need to add their training folders.

This feature may be added to spamtrainer in a future version.

Dec 6, 2012 1:36 AM in response to UptimeJeff

This sounds great!

Any chance that you might share this script with the community?

pterobyte

Level 6

11,098 points

Dec 6, 2012 2:40 AM in response to Jeff Hargrove

spamtrainer is already available for download at http://osx.topicdesk.com/spamtrainer

It is compatible with Mountain Lion. It allows you to run it at any time you like, can backup/restore your bayes DB and purge learned messages.

The additional features mentioned by UptimeJeff will be added in a future release.

Kostas B

Level 1

94 points

Dec 6, 2012 4:09 AM in response to pterobyte

Alex,

Are there any instructions for setting this up?

Best regards

Kostas

pterobyte

Level 6

11,098 points

Dec 6, 2012 4:43 AM in response to Kostas B

Kostas,

once installed, "spamtrainer -h" or "man spamtrainer" will show you the available parameters. You will also find a PDF document on your desktop.

HTH,

Alex

Kostas B

Level 1

94 points

Dec 6, 2012 4:46 AM in response to pterobyte

Thank you Alex!

Best

Kostas

Dec 7, 2012 3:05 AM in response to pterobyte

Hi,

I installed this today and get errors when using the -d option as follows:

server02:db admin$ sudo /usr/sbin/spamtrainer -d

Password:

/usr/sbin/spamtrainer: line 1625: /usr/bin/cvt_mail_data: No such file or directory

/usr/sbin/spamtrainer: line 1626: /usr/bin/cvt_mail_data: No such file or directory

Starting spamtrainer...

Training from user folders

Learning SPAM...

/usr/sbin/spamtrainer: line 1376: cd: /Library/Server/Mail/Data/mail//cur: No such file or directory

/usr/sbin/spamtrainer: line 1391: cd: /Library/Server/Mail/Data/mail//new: No such file or directory

Learned new SPAM (junk mail)

Learning HAM...

/usr/sbin/spamtrainer: line 1435: cd: /Library/Server/Mail/Data/mail//cur: No such file or directory

/usr/sbin/spamtrainer: line 1450: cd: /Library/Server/Mail/Data/mail//new: No such file or directory

Learned new HAM (not junk mail)

Syncing SpamAssassin Database

Displaying SpamAssassin Database Stats

0.000 0 14 0 non-token data: spam

0.000 0 63 0 non-token data: ham

Deleting learned HAM and SPAM

Done!

Output produced by spamtrainer Version 1.9.8

The database doesn't appear to be updated and the junk / notjunk mail is not deleted.

Appears to run OK and updates the database if run with no options.

This is with a standard mail setup (nothing moved to different locatins, junkmail and notjunkmail users exist and mailboxes can be accessed through the Mail client.

Any thoughts??

pterobyte

Level 6

11,098 points

Dec 7, 2012 3:45 AM in response to Chris Goodwin

What puzzles me is that you say it runs fine without the -d flag. The output you show is from a section where the -d flag doesn't count.

Looks like a path problem to me. Any chance you have spaces in volume or path names somewhere?

Could you mail me your dovecot configuration files?

Dec 7, 2012 3:59 AM in response to pterobyte

Yep - space in the boot volume name (Server HD) as delivered. Is renaming that going to have any effects on any services?

Which dovecot files in particular are you referring to? (I'm still trying to find my way around the new architecture having move from using Snow Leopard Server - new setup not migration).

pterobyte

Level 6

11,098 points

Dec 7, 2012 4:04 AM in response to Chris Goodwin

Space in Boot volume path is not a problem usually.

I think I found an obscure bug. Can you try and run spamtrainer again. You should be prompted to update to 1.9.9 which should fix it. Sorry about that and thanks for the feedback.

HTH,

Alex

P.S. Don't need it anymore, but for reference, the conf files are in /Library/Server/Mail/Config/dovecot/

Dec 7, 2012 4:14 AM in response to pterobyte

That seems to have done the trick!!

Excellent service (as usual).

Just need tosetup the auto run and hopefully good to go.

Many thanks,

Chris

Kostas B

Level 1

94 points

Dec 7, 2012 4:43 AM in response to Chris Goodwin

Sorry for this, but I need some clarification:

"When run, it will read the designated HAM and SPAM mailboxes, update the SpamAssassin databases and delete mail that has been learned from".

Suppose I create these mailboxes using: spamtrainer -p mymailpartition -j myjunkmailbox -n mynotjunkmailbox -d

1. How the users are aware of these mailboxes? Or is this the messages confirmed as "Junk" in their Mail client.

2. What -d (delete processed mail) means?

3. I have set all the marked as junk to redirect to a specific mail account and the users check this mailbox for any false marked items. Can they "whitelist" addresses from there?

Best regards

Kostas

pterobyte

Level 6

11,098 points

Dec 7, 2012 8:58 AM in response to Kostas B

spamtrainer will not create any mailboxes for you. The -p -j and -n parameters are only so you can point spamtrainer to different accounts. spamtrainer makes the same assumptions Apple's learn_junk_mail script does. It also uses the same backend software - spamassassin's sa-learn.

It is simply more flexible because you can give it different parameters if you have customised your system. It also allows you to delete spam and ham messages that you put in the junkmail and notjunkmail mailboxes after it has learned those messages.

Running spamtrainer or learn_junk_mail will train the bayes DB. It is not a whitelist. Whitelisting can be done in amavisd.conf or a configuration file like local.cf

HTH,

Alex

bdemore

Level 1

0 points

Dec 9, 2012 4:46 PM in response to essandess

Oddly when I run "sudo serveradmin settings mail:imap | grep junk" on a brand new 10.8 Server install, I receive the following:

mail:imap:junk_mail_userid = ""

mail:imap:not_junk_mail_userid = ""

I've tried to set them manually with the following command:

sudo serveradmin settings mail:imap:junk_mail_userid = "junkmail"

This command does not seem to change the status at all. Any advice?

Thanks in advance.

-Brian

pterobyte

Level 6

11,098 points

Dec 10, 2012 2:45 AM in response to bdemore

That would be the userid not the name. In any case you don't need to worry about this. Just make sure you have the accounts named junkmail and notjunkmail and feed them with spam and ham. Both learn_junk_mail and spamtrainer will find the mailbox and the id and train from those mailboxes.

Spamassassin Junkmail and Notjunkmail Learning in Mountain Lion