How can I test if SpamAssassin is working properly?

Set up a valid temporary email address and post it out to a couple of websites, and wait a couple of hours. You'll soon be receiving a wide variety of examples. This certainly might look like a 🙂 but I'll get back to this.

As for testing, send yourself some mail containing some obviously spammy phrases and words. Ensure references to the V* Rx product and the C* Rx product are included, for instance.

Client-based Baysian filters have an advantage over server-based Baysian filters, in that the client filters can dial into a particular user's email traffic. Do collect up your good mail and your spam mail, and use this to train your filters.

SpamAssassin (spamassassin.apache.org) typically doesn't re-route any arriving spam by default IIRC, it marks mail as being spammy and passes it through. You can alter this setting in Server Admin for Mail in the filtering mechanism settings. You can also use the Sieve tool to process the markings left by the SpamAssassin processing, and use that to route mail at the client level.

You mention SpamTrainer from (I assume) http://osx.topicdesk.com/spamtrainer -- have you looked at using the sa-learn mechanisms directly, and using these to train the filters with a pile of good mail and a second pile of spam. As stuff gets through, collect up more good and bad mail, and re-train the filters. It can take a couple of hundred messages, or more, to get a decent hit rate.

Do also ensure you have the relay blocking mechanisms enabled. Available lists (and related tools) such as the Spamhaus Exploits Block List (XBL) can be used to detect and thwart spam originating from known spam engines.

As for my original semi-humorous suggestion on posting up an email address, you can have the junkmail user be a real mailbox, post that address somewhere where only the bots will find the address, and have a free supply of incoming spam messages routed there. This can be used in turn to train the filters per the automatic mechanisms.

There are related intros available at:

http://www.afp548.com/articles/#email
http://www.afp548.com/article.php?story=20041104095414942
http://www.peachpit.com/articles/article.aspx?p=653382&seqNum=2&rl=1
http://osx.topicdesk.com/content/view/38/62/

among other sites, other resources such as the Apple manuals, and some available books. Some of the material at afp548 can be a bit dated (but can still provide a good overview); do check and confirm the version(s) referenced when reading.

Hey Garner,

Your situation can be improved. Your SpamAssassin set up should be working better for you.

First, if you just want to check and see if SpamAssassin is "working," you can check your amavis.log. A quick was to do this is to open ServerAdmin and then click Mail --> Logs and choose "Junk Mail/Virus Scanning." This log will show you what amavis is up to. At various times, you'll be able to see when amavis hands off a message to SpamAssassin and see the results (look for references to "SA msg read", "SA msg parse", etc.).

If you want to get some statistics you can show your boss, you may want to look into MailGraph. See Alex's tutorial here: http://osx.topicdesk.com/content/view/87/62/

However, I suspect you are worried that SpamAssassin is not learning and is not doing a better job of filtering out junk. Have you done the symlink fix, installed custom SARES rules, and tightened up your SA tag levels?

If not ... read on ...

Given your post, it's hard to tell how much customization you have done. Assuming you have done no more customization than you have shared in your post, then the following instructions should help.

You say you have already installed Spamtrainer. Check you have done the following:

(#1) Run:

sudo spamtrainer -f

This will Fix the amavisd/spamassassin configuration on OS X 10.4.x (in particular, this applies the symlink fix for Mac Server that needs to be made for proper SpamAssassin learning).

(#2) Then, run ...

sudo spamtrainer -a

This will add some additional SARES rule-sets to your configuration.

(#3) Then run ...

sudo spamtrainer -i

... and set up are daily cron script for SpamAssassin learning.

Once you have run these steps, be aware that SpamAssassin needs to learn at least 200 junk/notjunk e-mails before it kicks in. You will not see any impact from the learning process until you've fed SpamAssassin with at least 200 e-mails).

(#4) You need to optimize the SpamAssassin levels in your amavisd.conf file. Review this post by Pterobyte that outlines some suggested levels and make the appropriate changes for your client's system.

(#5) Finally, if you are using OS X 10.4 Server and you want to lock down down your Postfix restrictions, download Pterobyte's "Frontline Spam Defense" at:

http://osx.topicdesk.com/tutorials

This will cut down on more than 50% of the spam hitting your server before it ever gets to the content filter. +If you do this tutorial, please follow it exactly, I can't tell you how many people have downloaded that tutorial and then posted a problem to this board that is immediately resolved by pointing out a typo.+ 🙂

(#6) Once you have done all of this, it's really just a matter of letting SpamAssassin learn. Have you set up the junkmail and notjunkmail folders that allow users to feel SA ham and spam messages?

What text editor are you using the edit the file?

If you are using vi, something like ...

sudo vi /etc/postfix/main.cf

... should let you edit and save the changes.

You don't need to have mail services stopped to edit the file; however, you will need to stop and restart mail after you have saved the changes. That way, the new settings will be loaded.

Really important, make a backup of main.cf before you start work on that file.

As an alternative to vi editor, 'pico' is a lot simpler (as an introduction to editing) and has an on-screen help section...

sudo pico /etc/postfix/main.cf

You would make an initial backup of the file using the unix "cp" command...

sudo cp /etc/postfix/main.cf /etc/postfix/main.cf.bak

And re-instate the backup if required by reversing the two file arguments.

To save typeing the initial "/etc/postfix"... move to this directory with...
cd /etc/postfix

and then you just need... sudo cp main.cf main.cf.bak

(The "sudo" tells the shell to run the command as 'superuser' = root, which fixes the permissions problems).

Another Terminal tip... try typeing "cd /etc/post" then hit your 'tab' key... it should expand the "post" into the only available option... "postfix". If there was more than one option available, it will beep at you, so hit tab key again and it will list the options available.

I know that starting to edit files manually can be worrying, but it opens a lot of potential for the system. Just always have a backup of the file you are editing, and do things a step at a time, so you can revert back if need be.

-david

and (since I'm on a roll 🙂 ...

although you do not have to have postfix stopped whilst editing main.cf, do NOT have serveradmin open at the same time (in some types of config changes, this can lead to serveradmin re-writing manual changes when the two get 'out of sync' - best to keep SA quit).

You can tell postfix to reload its main.cf, without stopping/starting mail, by issuing "sudo postfix reload" in Terminal. You can stop and start mail services completely using "sudo serveradmin stop mail" and "sudo serveradmin start mail".

If you want more info on any of the unix commands, type "man command", e.g., "man cp" or look it up in this on-line man page...
http://developer.apple.com/documentation/Darwin/Reference/ManPages/index.html#// apple_ref/doc/framework/manpages

-david

In the immediate term, crank up the spam sensitivity threshold, or disable the filters.

Did you train the filters with several hundred good mail messages?

Garner wrote:
I checked the /var/virusmails folder and found an email that I sent from my hotmail account to my internal mail. It had this in it:

*X-Spam-Status: Yes, hits=3.559 tag=-999 tag2=3 kill=3 tests=BAYES_60,*
*HTML 90100, HTML_MESSAGE*
*X-Spam-Level: **

So is this something I can adjust by editing a file somewhere? I sure hope so.

Server Admin -> Mail-> Settings-> Filters

It looks from the above that you have Minimum Junk Mail Score= 3, and that the junk mail messages be: Deleted. This is an exeptionally low level!!!!

I suggest that you raise the Minimum Junk Mail Score to about 4 and change the action to: Delivered.

Without doing manual editing, your only real choice is to tag suspected spam and let it be delivered. If you really want to delete some spam (and not deliver it) then you need to set the min score far higher, like 8 or 10.

-david

Garner wrote:
Yes, I had around 200 good emails, but come to think of it, they were mostly internal addresses.

How could I get several hundred good mail messages? Do I need to have a bunch of people redirect good external messages to the notjunkmail address?

Your spamassassin IS learning correctly. From your previous post you can see that there was a test: tests=BAYES_60. This would not have been triggered if your spamassassin had not been learning and if you had not passed the minimum threshold of 200 spam and 200 non-spam.

Spamassassin does not actually need to be fed any mail; it will self-learn. The default threshold scores for self-learning are +15 (for spam) and -2 (for non-spam). These can be changed (by editing a file!) but, counter-intuitively, you are actually better building up a large body of non-spam because the range of words used in your non-spam are likely to be far smaller and more specific than the range of words used in spam. This means that the bayes analysis is more likely to be able to distinguish good mail, and therefore differentiate it from other mail (likely spam).

Please also note that spam training only affects the bayesian analysis part of spamassassin - it is only one of many tests that spamassassin uses to analyse mail (see some of the other tests in the mail header of your previous post). It is also very slow to learn from any mistakes it makes - if you have a corpus of the minimum 200 messages then 1 extra is only going to sway the result a little. When you get up to the thousands, even less so. Hence, the real reduction in spam will only come when you feel confident enough to implement the osx.topicdesk "Frontline Defense..." doc and implement other manual 'tweaks'. Alternatively, you can always contact the osx.topicdesk directly and I'm sure Alex (who posts in here as Pterobyte) will be glad to assist (in a professional capacity 😉 May be worth it to your boss!

-david

Garner ... This is a shot in the dark, and it only applies if David_x's suggestion doesn't work and you are not getting any mail to the clients. Check your Mail Queue and see if the 200 "good" e-mails you mention are stacking up there. If something you did throw off amavis, you might find that the mails are just stuck in the queue because amavis is having issues. If that's the case, post back here and let us know ... we just need to track down the trouble so that we can get those e-mails to the clients.

Message was edited by: Joel Mcintosh1