SpamAssassin & Razor how-to
SpamAssassin & Razor how-to:
When used in conjuction with Vipul's Razor, SpamAssassin's accuracy can really go to new levels, making junkmail filtering far more accurate. Here's how to set both of these up on a Tiger server system (note: be sure to backup your system, or at least make copies of any files you're modifying before starting!):
First, fix a problem with SpamAssassin's out of the box configuration, by re-creating a symlink. I've found it best to sudo to root, then su to clamav, to get the permissions correct, as follows (warning: this will remove the current database of 'learned junk' if it exists):
sudo -s
cd /var/amavis
mv .spamassassin .spamassassin.old
su clamav
ln -s /var/clamav/.spamassassin /var/amavis/.spamassassin
When you've done this, do 'ls -la' and you should see a line like this:
lrwxr-xr-x 1 clamav clamav 25 Feb 8 21:51 .spamassassin -> /var/clamav/.spamassassin
Next, check a few parameters in the '/etc/mail/spamassassin/local.cf' file. First run:
spamassassin --lint
If you're running 10.4.2 - 10.4.4, you might see output similar to:
config: SpamAssassin failed to parse line, skipping: auto_learn 1
config: SpamAssassin failed to parse line, skipping: safe_reporting 0
config: SpamAssassin failed to parse line, skipping: use tersereport 0
config: SpamAssassin failed to parse line, skipping: subject_tag * Warning: Junk Mail *
config: SpamAssassin failed to parse line, skipping: rewrite_subject 0
lint: 5 issues detected. please rerun with debug enabled for more information.
To fix this, first backup your '/etc/mail/spamassassin/local.cf' file, then open the file and change any of the above output, as follows:
change 'auto_learn' to 'bayes autolearn'
change 'safe_reporting' to 'safe_report'
comment out 'use tersereport 0'
comment out 'subject_tag' and 'rewrite_subject'
Now issuing 'spamassassin --lint' should just return you to your command prompt without any output. If so, you're all set.
Next, we need to check a bunch of things in the '/etc/amavisd.conf' file. First, backup your file, then make sure the following lines look as follows (don't forget the semi-colons at the end of each line!):
$sa local_testsonly = 0;
I like to set this to 0 (disable), in order to enable network tests by tools such as razor, which can dramatically improve SpamAssassin's accuracy. See below for more details on using Razor.
$sa tag_leveldeflt = -999;
This low threshold causes SpamAssassin to add X-Spam headers to all local messages scoring above -999, which should be all of them.
$sa tag2_leveldeflt = 5;
This is the SpamAssassin score at which to consider a message spam and tag it as such (by modifying the subject line), a good starting point is about 5
$sa kill_leveldeflt = 10;
If a message's SpamAssassin score reaches this level, then the 'final spamdestiny' action is taken. I like to set this to around 10, depending on what 'final spamdestiny' is. Remember, if 'final spamdestiny' is set to reject mail, this is the score at which the mail will be rejected. I like to start out with the 'D_PASS' setting for starters, which delivers the mail even if this score is reached (see below for more info on the 'final spamdestiny' setting).
$sa spam_subjecttag = ' * Junk Mail *';
When a message's SpamAssassin score reaches the '$sa tag2_leveldeflt' score (above), then the text here is added to the beginning of the email subject line.
$final spamdestiny = [D DISCARD,D_BOUNCE,DPASS];
This tells SpamAssassin what to do with a message that reaches the '$sa kill_leveldeflt' score. It can be set to one of the three values above (so, for example, to deliver all mail, the line would be $final spamdestiny = D_PASS;).
@local domainsacl = [acl]
This line tells SpamAssassin which email domains are hosted by the local system, and thus which ones to take action on. Make sure all possible local domains that will be handled by your mail server are listed here. An example would be:
@local domainsacl = ( '.myDomain1.com', '.myDomain2.com', '.myDomain3.com' );
OK: next, if you run virtual domains, make sure this line is uncommented (i.e. no '#' sign at the beginning of the line):
@local domainsmaps = ( 1 );
If the line doesn't exist and you need it, add it after the '@local domainsacl = ...' line.
If you want to use razor (see below), make sure to add the following lines, if they don't already exist:
use_razor2 1
razor_timeout 10
score RAZOR2 CF_RANGE_51100 4.0
Whew! OK we're done with the '/etc/amavisd.conf' file. Next, you need to feed junkmail to your system, and lots of it. First, you'll need to set up two 'training' accounts on your system. One called 'junkmail' and one called 'notjunkmail.' This is explained in more detail on page 52 of the Mail Service Admin guide, available here: http://images.apple.com/server/pdfs/MailServicev10.4.pdf.
After you've set up these accounts, you need to redirect (be sure not to forward!!) around 200 junk and non-junk messages (around 400 total) to each mailbox. This is crucial, and if there aren't enough messages in there, SpamAssassin will be pretty inaccurate. One way to make it easier to feed the boxes is to setup the two IMAP accounts in your mail program. Then you can just drag and drop junk mail from your real account into the inbox for the 'junkmail' account (and likewise for the 'notjunkmail' account). This is a lot easier than the mass redirects. If appropriate, do the same for other users on your system.
One more thing that is sometimes broken out of the box with Tiger Server is the script that trains the junkmail filter each day (as described in the admin guide from step 4), using the contents of the 'junkmail' and 'notjunkmail' accounts. You'll need to fix this for automatic training of the filter, and an easy way to do this is to the use the great SpamTrainer for OS X, which is available here: http://osx.topicdesk.com/downloads/. Just download and follow the simple instructions, and your training scripts will be all set to run.
Improving SpamAssasin's accuracy with Razor:
There are many add-ons that can be enabled which really beef up SpamAssassin's accuracy, and one I like to use is Vipul's Razor. In order to install this, you'll need the Xcode 2.2 developer tools installed on your system. To get Xcode visit the Apple Developer's site, register for a free account, then download and install the tools. Once you've done that, download razor-agents the the razor-agents sdk here. Once downloaded, unpack them like so:
tar -xvjf razor-agents-2.77.tar.bz2
tar -xvjf razor-agents-sdk-2.07.tar.bz2
Then cd into the razor agents sdk dir and install, like so:
cd razor-agents-sdk-2.07
perl Makefile.PL
make
make test
make install
After you've done SDK, cd into the razor agents dir and do likewise, like so:
cd ../razor-agents-2.77
perl Makefile.PL
make
make test
make install
OK - now to set up Razor: if you're still in a root shell (i.e. you see a '#' sign at the end of your shell prompt), then type 'exit' and hit enter to return to your regular user shell (if you're not sure, just open a new terminal window, or ssh session if you're doing this over ssh). Then run:
razor-admin -create
which creates a config file in your user home directory. Then choose an email address with which to register razor, and do the registration:
razor-admin -register -user=user@domain.com
That should do it. Once you've completed these steps and trained SA, your filtering will be much more accurate.
iBook G4 Mac OS X (10.4.4)
When used in conjuction with Vipul's Razor, SpamAssassin's accuracy can really go to new levels, making junkmail filtering far more accurate. Here's how to set both of these up on a Tiger server system (note: be sure to backup your system, or at least make copies of any files you're modifying before starting!):
First, fix a problem with SpamAssassin's out of the box configuration, by re-creating a symlink. I've found it best to sudo to root, then su to clamav, to get the permissions correct, as follows (warning: this will remove the current database of 'learned junk' if it exists):
sudo -s
cd /var/amavis
mv .spamassassin .spamassassin.old
su clamav
ln -s /var/clamav/.spamassassin /var/amavis/.spamassassin
When you've done this, do 'ls -la' and you should see a line like this:
lrwxr-xr-x 1 clamav clamav 25 Feb 8 21:51 .spamassassin -> /var/clamav/.spamassassin
Next, check a few parameters in the '/etc/mail/spamassassin/local.cf' file. First run:
spamassassin --lint
If you're running 10.4.2 - 10.4.4, you might see output similar to:
config: SpamAssassin failed to parse line, skipping: auto_learn 1
config: SpamAssassin failed to parse line, skipping: safe_reporting 0
config: SpamAssassin failed to parse line, skipping: use tersereport 0
config: SpamAssassin failed to parse line, skipping: subject_tag * Warning: Junk Mail *
config: SpamAssassin failed to parse line, skipping: rewrite_subject 0
lint: 5 issues detected. please rerun with debug enabled for more information.
To fix this, first backup your '/etc/mail/spamassassin/local.cf' file, then open the file and change any of the above output, as follows:
change 'auto_learn' to 'bayes autolearn'
change 'safe_reporting' to 'safe_report'
comment out 'use tersereport 0'
comment out 'subject_tag' and 'rewrite_subject'
Now issuing 'spamassassin --lint' should just return you to your command prompt without any output. If so, you're all set.
Next, we need to check a bunch of things in the '/etc/amavisd.conf' file. First, backup your file, then make sure the following lines look as follows (don't forget the semi-colons at the end of each line!):
$sa local_testsonly = 0;
I like to set this to 0 (disable), in order to enable network tests by tools such as razor, which can dramatically improve SpamAssassin's accuracy. See below for more details on using Razor.
$sa tag_leveldeflt = -999;
This low threshold causes SpamAssassin to add X-Spam headers to all local messages scoring above -999, which should be all of them.
$sa tag2_leveldeflt = 5;
This is the SpamAssassin score at which to consider a message spam and tag it as such (by modifying the subject line), a good starting point is about 5
$sa kill_leveldeflt = 10;
If a message's SpamAssassin score reaches this level, then the 'final spamdestiny' action is taken. I like to set this to around 10, depending on what 'final spamdestiny' is. Remember, if 'final spamdestiny' is set to reject mail, this is the score at which the mail will be rejected. I like to start out with the 'D_PASS' setting for starters, which delivers the mail even if this score is reached (see below for more info on the 'final spamdestiny' setting).
$sa spam_subjecttag = ' * Junk Mail *';
When a message's SpamAssassin score reaches the '$sa tag2_leveldeflt' score (above), then the text here is added to the beginning of the email subject line.
$final spamdestiny = [D DISCARD,D_BOUNCE,DPASS];
This tells SpamAssassin what to do with a message that reaches the '$sa kill_leveldeflt' score. It can be set to one of the three values above (so, for example, to deliver all mail, the line would be $final spamdestiny = D_PASS;).
@local domainsacl = [acl]
This line tells SpamAssassin which email domains are hosted by the local system, and thus which ones to take action on. Make sure all possible local domains that will be handled by your mail server are listed here. An example would be:
@local domainsacl = ( '.myDomain1.com', '.myDomain2.com', '.myDomain3.com' );
OK: next, if you run virtual domains, make sure this line is uncommented (i.e. no '#' sign at the beginning of the line):
@local domainsmaps = ( 1 );
If the line doesn't exist and you need it, add it after the '@local domainsacl = ...' line.
If you want to use razor (see below), make sure to add the following lines, if they don't already exist:
use_razor2 1
razor_timeout 10
score RAZOR2 CF_RANGE_51100 4.0
Whew! OK we're done with the '/etc/amavisd.conf' file. Next, you need to feed junkmail to your system, and lots of it. First, you'll need to set up two 'training' accounts on your system. One called 'junkmail' and one called 'notjunkmail.' This is explained in more detail on page 52 of the Mail Service Admin guide, available here: http://images.apple.com/server/pdfs/MailServicev10.4.pdf.
After you've set up these accounts, you need to redirect (be sure not to forward!!) around 200 junk and non-junk messages (around 400 total) to each mailbox. This is crucial, and if there aren't enough messages in there, SpamAssassin will be pretty inaccurate. One way to make it easier to feed the boxes is to setup the two IMAP accounts in your mail program. Then you can just drag and drop junk mail from your real account into the inbox for the 'junkmail' account (and likewise for the 'notjunkmail' account). This is a lot easier than the mass redirects. If appropriate, do the same for other users on your system.
One more thing that is sometimes broken out of the box with Tiger Server is the script that trains the junkmail filter each day (as described in the admin guide from step 4), using the contents of the 'junkmail' and 'notjunkmail' accounts. You'll need to fix this for automatic training of the filter, and an easy way to do this is to the use the great SpamTrainer for OS X, which is available here: http://osx.topicdesk.com/downloads/. Just download and follow the simple instructions, and your training scripts will be all set to run.
Improving SpamAssasin's accuracy with Razor:
There are many add-ons that can be enabled which really beef up SpamAssassin's accuracy, and one I like to use is Vipul's Razor. In order to install this, you'll need the Xcode 2.2 developer tools installed on your system. To get Xcode visit the Apple Developer's site, register for a free account, then download and install the tools. Once you've done that, download razor-agents the the razor-agents sdk here. Once downloaded, unpack them like so:
tar -xvjf razor-agents-2.77.tar.bz2
tar -xvjf razor-agents-sdk-2.07.tar.bz2
Then cd into the razor agents sdk dir and install, like so:
cd razor-agents-sdk-2.07
perl Makefile.PL
make
make test
make install
After you've done SDK, cd into the razor agents dir and do likewise, like so:
cd ../razor-agents-2.77
perl Makefile.PL
make
make test
make install
OK - now to set up Razor: if you're still in a root shell (i.e. you see a '#' sign at the end of your shell prompt), then type 'exit' and hit enter to return to your regular user shell (if you're not sure, just open a new terminal window, or ssh session if you're doing this over ssh). Then run:
razor-admin -create
which creates a config file in your user home directory. Then choose an email address with which to register razor, and do the registration:
razor-admin -register -user=user@domain.com
That should do it. Once you've completed these steps and trained SA, your filtering will be much more accurate.
iBook G4 Mac OS X (10.4.4)
