essandess

Q: Spamassassin Junkmail and Notjunkmail Learning in Mountain Lion

Like many others <https://discussions.apple.com/thread/4181320?start=0&tstart=0>, I wondered what happened to spamassassin and its Bayesian learning mailboxes "junkmail" and "notjunkmail" in OS X Server. Perhaps Apple had automated this process? There certainly isn't any documentation. I didn't think much of it until spam began falling into my server's INBOX, whereupon looking for a fix I discovered that spamassassin learning capabilities still work in Mountain Lion OS X Server, but they aren't automated in any way.

 

Here's how to train your server to detect the junk mail you receive, beyond the appropriate spam filter and greylist settings Server.app>Mail.

 

First, confirm that your serveradmin settings know about the spamassassin Bayesian learning mailboxes "junkmail" and "notjunkmail":

 

sudo serveradmin settings mail:imap | grep junk

 

mail:imap:junk_mail_userid = "junkmail"

mail:imap:not_junk_mail_userid = "notjunkmail"

 

Then use Server.app to add the network accounts "junkmail" and "notjunkmail" and edit access so that these accounts only have access to the Mail service. Add these accounts to Mail so that you (and other trusted users) have access to them.

 

Then dump all your junkmail from your server accounts and other cloud email accounts into the "junkmail" mailbox. Move any messages that aren't junk but marked as such into the mailbox "notjunkmail".

 

Finally,

 

sudo /Applications/Server.app/Contents/ServerRoot/usr/libexec/spamassassin/learn_jun k_mail.sh

 

This will train spamassassin based on the content of these mailboxes, and hopefully you will begin to see amavisd properly mark junkmail as "** JUNK MAIL **" in the subject line.

Mac mini Server (Mid 2010), OS X Mountain Lion (10.8.2), Lion Server, EyeTV HD, Turbo.264 HD

Posted on Nov 4, 2012 2:40 AM

Close

Q: Spamassassin Junkmail and Notjunkmail Learning in Mountain Lion

  • All replies
  • Helpful answers

Previous Page 2 of 3 last Next
  • by Jeff Hargrove,

    Jeff Hargrove Jeff Hargrove Nov 9, 2012 7:20 PM in response to essandess
    Level 2 (235 points)
    Nov 9, 2012 7:20 PM in response to essandess

    This is really helpful!

     

    I was reading in Apple's advanced server admin manual that it was a default that the server will train junk mail at 2:15. Another let down ...

     

    I think you should be writing the unofficial manual for all of us novices.

     

    Any chance I could get your help about my quarantine problem? I think it is trying to deliver to the quarantine address, but fails, but don't know why.

     

    Again thanks for your time is getting spam training set up.

     

    Cheers

     

    Jeff

  • by UptimeJeff,

    UptimeJeff UptimeJeff Dec 5, 2012 7:29 PM in response to essandess
    Level 4 (3,477 points)
    Dec 5, 2012 7:29 PM in response to essandess

    I've never been happy with bayes training since the old cyrus shared mailbox days.

     

    Users shouldn't need to redirect messages and sharing access to junkmail/notjunkmail imap accounts is cumbersome.

     

    Today, I tackled this with a quick script which does this:

     

    - Scans every mail user for 2 folders (junkmail/notjunkmail)

    - Feeds those messages to sa-learn, to train the bayes filters.

     

    I will add

    - purge the user junkmail/notjunkmail folders of messages > 30 days old.

     

    End result, users simply drag messages within their own imap account.

    The script can specifcy the standard 'Junk' folder so that client apps that filter to 'Junk' will contribute to the common bayes DB.

     

    This creates an easy to use training system which allows all users to contribute easily.

    Requires no server config, users only need to add their training folders.

     

    This feature may be added to spamtrainer in a future version.

  • by Jeff Hargrove,

    Jeff Hargrove Jeff Hargrove Dec 6, 2012 1:36 AM in response to UptimeJeff
    Level 2 (235 points)
    Dec 6, 2012 1:36 AM in response to UptimeJeff

    This sounds great!

     

    Any chance that you might share this script with the community?

  • by pterobyte,

    pterobyte pterobyte Dec 6, 2012 2:40 AM in response to Jeff Hargrove
    Level 6 (11,101 points)
    Servers Enterprise
    Dec 6, 2012 2:40 AM in response to Jeff Hargrove

    spamtrainer is already available for download at http://osx.topicdesk.com/spamtrainer 

    It is compatible with Mountain Lion. It allows you to run it at any time you like, can backup/restore your bayes DB and purge learned messages.

    The additional features mentioned by UptimeJeff will be added in a future release.

  • by Kostas B,

    Kostas B Kostas B Dec 6, 2012 4:09 AM in response to pterobyte
    Level 1 (90 points)
    Dec 6, 2012 4:09 AM in response to pterobyte

    Alex,

     

    Are there any instructions for setting this up?

     

    Best regards

     

    Kostas

  • by pterobyte,

    pterobyte pterobyte Dec 6, 2012 4:43 AM in response to Kostas B
    Level 6 (11,101 points)
    Servers Enterprise
    Dec 6, 2012 4:43 AM in response to Kostas B

    Kostas,

     

    once installed, "spamtrainer -h" or "man spamtrainer" will show you the available parameters. You will also find a PDF document on your desktop.

     

    HTH,

    Alex

  • by Kostas B,

    Kostas B Kostas B Dec 6, 2012 4:46 AM in response to pterobyte
    Level 1 (90 points)
    Dec 6, 2012 4:46 AM in response to pterobyte

    Thank you Alex!

     

    Best

     

    Kostas

  • by Chris Goodwin,

    Chris Goodwin Chris Goodwin Dec 7, 2012 3:05 AM in response to pterobyte
    Level 1 (59 points)
    Dec 7, 2012 3:05 AM in response to pterobyte

    Hi,

     

    I installed this today and get errors when using the -d option as follows:

     

    server02:db admin$ sudo /usr/sbin/spamtrainer -d

    Password:

     

     

    /usr/sbin/spamtrainer: line 1625: /usr/bin/cvt_mail_data: No such file or directory

    /usr/sbin/spamtrainer: line 1626: /usr/bin/cvt_mail_data: No such file or directory

     

     

    Starting spamtrainer...

     

     

    Training from user folders

    Learning SPAM...

    /usr/sbin/spamtrainer: line 1376: cd: /Library/Server/Mail/Data/mail//cur: No such file or directory

    /usr/sbin/spamtrainer: line 1391: cd: /Library/Server/Mail/Data/mail//new: No such file or directory

    Learned new SPAM (junk mail)

    Learning HAM...

    /usr/sbin/spamtrainer: line 1435: cd: /Library/Server/Mail/Data/mail//cur: No such file or directory

    /usr/sbin/spamtrainer: line 1450: cd: /Library/Server/Mail/Data/mail//new: No such file or directory

    Learned new HAM (not junk mail)

    Syncing SpamAssassin Database

    Displaying SpamAssassin Database Stats

    0.000          0         14          0  non-token data: spam

    0.000          0         63          0  non-token data: ham

    Deleting learned HAM and SPAM

    Done!

    Output produced by spamtrainer Version 1.9.8

     

     

    The database doesn't appear to be updated and the junk / notjunk mail is not deleted.

     

    Appears to run OK and updates the database if run with no options.

     

    This is with a standard mail setup (nothing moved to different locatins, junkmail and notjunkmail users exist and mailboxes can be accessed through the Mail client.

     

    Any thoughts??

  • by pterobyte,

    pterobyte pterobyte Dec 7, 2012 3:45 AM in response to Chris Goodwin
    Level 6 (11,101 points)
    Servers Enterprise
    Dec 7, 2012 3:45 AM in response to Chris Goodwin

    What puzzles me is that you say it runs fine without the -d flag. The output you show is from a section where the -d flag doesn't count.

     

    Looks like a path problem to me. Any chance you have spaces in volume or path names somewhere?

     

    Could you mail me your dovecot configuration files?

  • by Chris Goodwin,

    Chris Goodwin Chris Goodwin Dec 7, 2012 3:59 AM in response to pterobyte
    Level 1 (59 points)
    Dec 7, 2012 3:59 AM in response to pterobyte

    Yep - space in the boot volume name (Server HD) as delivered. Is renaming that going to have any effects on any services?

     

    Which dovecot files in particular are you referring to? (I'm still trying to find my way around the new architecture having move from using Snow Leopard Server - new setup not migration).

  • by pterobyte,

    pterobyte pterobyte Dec 7, 2012 4:04 AM in response to Chris Goodwin
    Level 6 (11,101 points)
    Servers Enterprise
    Dec 7, 2012 4:04 AM in response to Chris Goodwin

    Space in Boot volume path is not a problem usually.

     

    I think I found an obscure bug. Can you try and run spamtrainer again. You should be prompted to update to 1.9.9 which should fix it. Sorry about that and thanks for the feedback.

     

    HTH,

    Alex

     

    P.S. Don't need it anymore, but for reference, the conf files are in /Library/Server/Mail/Config/dovecot/

  • by Chris Goodwin,

    Chris Goodwin Chris Goodwin Dec 7, 2012 4:14 AM in response to pterobyte
    Level 1 (59 points)
    Dec 7, 2012 4:14 AM in response to pterobyte

    That seems to have done the trick!!

     

    Excellent service (as usual).

     

    Just need tosetup the auto run and hopefully good to go.

     

    Many thanks,

     

    Chris

  • by Kostas B,

    Kostas B Kostas B Dec 7, 2012 4:43 AM in response to Chris Goodwin
    Level 1 (90 points)
    Dec 7, 2012 4:43 AM in response to Chris Goodwin

    Sorry for this, but I need some clarification:

     

    "When run, it will read the designated HAM and SPAM mailboxes, update the SpamAssassin databases and delete mail that has been learned from".

     

    Suppose I create these mailboxes using: spamtrainer -p mymailpartition -j myjunkmailbox -n mynotjunkmailbox -d

     

    1. How the users are aware of these mailboxes? Or is this the messages confirmed as "Junk" in their Mail client.

     

    2. What -d (delete processed mail) means?

     

    3. I have set all the marked as junk to redirect to a specific mail account and the users check this mailbox for any false marked items. Can they "whitelist" addresses from there?

     

    Best regards

     

    Kostas

  • by pterobyte,

    pterobyte pterobyte Dec 7, 2012 8:58 AM in response to Kostas B
    Level 6 (11,101 points)
    Servers Enterprise
    Dec 7, 2012 8:58 AM in response to Kostas B

    spamtrainer will not create any mailboxes for you. The -p -j and -n parameters are only so you can point spamtrainer to different accounts. spamtrainer makes the same assumptions Apple's learn_junk_mail script does. It also uses the same backend software - spamassassin's sa-learn.

    It is simply more flexible because you can give it different parameters if you have customised your system. It also allows you to delete spam and ham messages that you put in the junkmail and notjunkmail mailboxes after it has learned those messages.

     

    Running spamtrainer or learn_junk_mail will train the bayes DB. It is not a whitelist. Whitelisting can be done in amavisd.conf or a configuration file like local.cf

     

    HTH,

    Alex

  • by bdemore,

    bdemore bdemore Dec 9, 2012 4:46 PM in response to essandess
    Level 1 (0 points)
    Dec 9, 2012 4:46 PM in response to essandess

    Oddly when I run "sudo serveradmin settings mail:imap | grep junk" on a brand new 10.8 Server install, I receive the following:

     

    mail:imap:junk_mail_userid = ""

    mail:imap:not_junk_mail_userid = ""

     

    I've tried to set them manually with the following command: 

     

    sudo serveradmin settings mail:imap:junk_mail_userid = "junkmail"

     

    This command does not seem to change the status at all.  Any advice?

     

    Thanks in advance.

     

    -Brian

Previous Page 2 of 3 last Next