Deliver Spam with subject heading and headers not working

I have begun the nasty process of training spamassassin. Originally it was setup to redirect all spam to a quarantine account. That worked fine but the since the filter has been refined too much good mail was getting redirected (along with a ton of junk). So I changed the option from redirect to deliver with the subject header to ****SPAM****. All mail, good and bad, would be delivered but none of the junk would be marked with the subject header nor would there be any X-SPAM headers at all. When I revert to the redirect option, the spam is then redirected fine again with X-SPAM STATUS and X-SPAM LEVEL headers. I did do some tinkering with some whitelist and blacklist comments in the local.cf and user-prefs files. Below is from my local.cf file which is now configured to redirect, anything look wrong here?

# This is the right place to customize your installation of SpamAssassin.
#
# See 'perldoc Mail::SpamAssassin::Conf' for details of what can be
# tweaked.
#
###########################################################################
#
# rewrite_subject 0
# report_safe 1
# trusted_networks 212.17.35.

# Bayesian Auto Learn
auto_learn 1

# Safe Reporting
safe_reporting 0

# Full/Terse Reporting
use tersereport 0

# Subject Tag
subject_tag ****SPAM****

# Rewrite the Subject
rewrite_subject 0

# Use Bayesian Filtering
use_bayes 1

# OK locals
ok_locales en

# OK languages
ok_languages en

# Required hits to be marked as spam
required_hits 4

# add_header all Checker-Version SpamAssassin

The "# add_header all Checker-Version SpamAssassin" was an attempt to have spamassassin put this header in all scanned messages which also doesn't show up in any messages.

Is there a way to just clear out or reset all the spamassasin settings and start from scratch?

Thanks.

XServe, Mac OS X (10.4.8)

Posted on Feb 6, 2007 7:56 AM

Reply
11 replies

Feb 6, 2007 8:33 AM in response to oscarthedog

For the most part, Amavis is going to handle most of want you do. It will use SpamAssassin for scoring, but amavis will process your mail. That's why your changes to local.cf are not working right -- they are being ignored.

So, you need to work in your amavisd.conf file to do what you want (/etc/adamisd.conf).

(1) Revert back to a backup of your local.cf file.

(2) Visit http://wiki.apache.org/spamassassin/SpamAssassinon_Mac_OS_XServer and scrol down to the section titled "Lint (the nasty stuff that grows between your toes)" and follow the instructions. They are not necessary (remember most of local.cf gets ignored), but they are nice and allow you to run "spamassassin --lint" and get clean results (when testing score modifications and whitelisting/blacklisting in local.cf ... which are not ignored).

(3) Read your amavisd.conf file (it is well commented, and pretty easy to get a handle on). Mostly, you want to work in "Section VII - External programs, virus scanners, SpamAssassin" (at least it's Section VII in my version of the file), and scroll to the "SpamAssassin settings" in that section. You want to set your tag, tag2, and kill levels properly there.

Then scroll down and look for "# string to prepend to Subject header field when message exceeds tag2 level" and set this section to look like this ...

$sa spam_subjecttag = ' * JUNK MAIL * '; # (defaults to undef, disabled)
# (only seen when spam is passed and recipient is
# in local_domains*)

$sa spam_modifiessubj = 1; # in @spam modifies_subjmaps, default is true

When you are done, stop and restart mail.

Feb 7, 2007 7:16 AM in response to Joel Mcintosh1

Joel:

Thanks for all the useful info.

Here is what my spamassassin settings now look like in amavisd.conf:

# SpamAssassin settings

# $sa local_testsonly is passed to Mail::SpamAssassin::new as a value
# of the option local testsonly. See Mail::SpamAssassin man page.
# If set to 1, no tests that require internet access will be performed.
#
$sa local_testsonly = 1; # (default: false)
#$sa autowhitelist = 1; # turn on AWL (default: false)

$sa mail_body_sizelimit = 64*1024; # don't waste time on SA if mail is larger
# (less than 1% of spam is > 64k)
# default: undef, no limitations

# default values, can be overridden by more specific lookups, e.g. SQL
$sa tag_leveldeflt = -999; # add spam info headers if at, or above that level
$sa tag2_leveldeflt = 6.0; # add 'spam detected' headers at that level
$sa kill_leveldeflt = 22.0;
#$sa kill_leveldeflt = $sa tag2_leveldeflt; # triggers spam evasive actions
# at or above that level: bounce/reject/drop,
# quarantine, and adding mail address extension
#
# The $sa tag_leveldeflt, $sa tag2_leveldeflt and $sa kill_leveldeflt
# may also be hashrefs to hash lookup tables, to make static per-recipient
# settings possible without having to resort to SQL or LDAP lookups.

# a quick reference:
# tag_level controls adding the X-Spam-Status and X-Spam-Level headers,
# tag2_level controls adding 'X-Spam-Flag: YES', and editing Subject,
# kill_level controls 'evasive actions' (reject, quarantine, extensions);
# it only makes sense to maintain the relationship:
# tag_level <= tag2_level <= kill_level

# string to prepend to Subject header field when message exceeds tag2 level
$sa spam_subjecttag = ' ****SPAM****'; # (defaults to undef, disabled)
# (only seen when spam is passed and recipient is
# in local_domains*)

$sa spam_modifiessubj = 1; # in @spam modifies_subjmaps, default is true

# Example: modify Subject for all local recipients except user@example.com
#$sa spam_modifiessubj = [qw( !user@example.com . )];

So, if all goes well, I should see messages with 6 or more hits delivered unmodified with the exception of the ****SPAM**** subject header, right? Or am I missing something else.

Feb 7, 2007 8:32 AM in response to oscarthedog

So, if all goes well, I should see messages with 6 or more hits delivered
unmodified with the exception of the ****SPAM**** subject header, right?
Or am I missing something else.


Between 6-22 will be tagged.
22+ depends on what your $final spamdestiny is set for (also in amavisd.conf)

kill level of 22 is very high for a trained server. My servers are generally set around 7-10 depending on how diligent the users are with training at the site.

Take a look at your final destiny settings. Reject and Bouncing will cause your server to deliver spam to forged sender addresses. Avoid using reject or bounce.

Also.. the default setting is to quarnatine spam and viruses to /var/viruses. This will eat space on your drive. If you don't want the quarantine directory, comment this line:
$QUARANTINEDIR = '/var/virusmails';
Take a look in that folder, you probably have thousands of files.

Jeff

Feb 8, 2007 9:51 PM in response to Joel Mcintosh1

Well I made the changes but I'm still not getting any results when the option in server admin is set to deliver with subject changed to ****SPAM**** with or without the message attached as MIME. I am seeing messages with hits way over my tag2 threshold getting delivered without any subject header or XSPAM headers. The only option that seems to work is the option to redirect it to a quarantine mailbox which is a bit of a pain as spamassassin is still in it's initial traing period.

Here are my spamassassin settings from amavisd.conf: # SpamAssassin settings

# $sa local_testsonly is passed to Mail::SpamAssassin::new as a value
# of the option local testsonly. See Mail::SpamAssassin man page.
# If set to 1, no tests that require internet access will be performed.
#
$sa local_testsonly = 1; # (default: false)
#$sa autowhitelist = 1; # turn on AWL (default: false)

$sa mail_body_sizelimit = 64*1024; # don't waste time on SA if mail is larger
# (less than 1% of spam is > 64k)
# default: undef, no limitations

# default values, can be overridden by more specific lookups, e.g. SQL
$sa tag_leveldeflt = -999; # add spam info headers if at, or above that level
$sa tag2_leveldeflt = 7.0; # add 'spam detected' headers at that level
$sa kill_leveldeflt = 22.0;
#$sa kill_leveldeflt = $sa tag2_leveldeflt; # triggers spam evasive actions
# at or above that level: bounce/reject/drop,
# quarantine, and adding mail address extension
#
# The $sa tag_leveldeflt, $sa tag2_leveldeflt and $sa kill_leveldeflt
# may also be hashrefs to hash lookup tables, to make static per-recipient
# settings possible without having to resort to SQL or LDAP lookups.

# a quick reference:
# tag_level controls adding the X-Spam-Status and X-Spam-Level headers,
# tag2_level controls adding 'X-Spam-Flag: YES', and editing Subject,
# kill_level controls 'evasive actions' (reject, quarantine, extensions);
# it only makes sense to maintain the relationship:
# tag_level <= tag2_level <= kill_level

# string to prepend to Subject header field when message exceeds tag2 level
$sa spam_subjecttag = ' ****SPAM****'; # (defaults to undef, disabled)
# (only seen when spam is passed and recipient is
# in local_domains*)

$sa spam_modifiessubj = 1; # in @spam modifies_subjmaps, default is true

# Example: modify Subject for all local recipients except user@example.com
#$sa spam_modifiessubj = [qw( !user@example.com . )];


Anything I'm missing? I noticed there is a amavisd.conf.personal file also in etc. Should I be tweaking that as well?

Thanks.

Feb 9, 2007 3:03 AM in response to oscarthedog

Well I made the changes but I'm still not getting any
results when the option in server admin is set to
deliver with subject changed to ****SPAM**** with
or without the message attached as MIME. I am seeing
messages with hits way over my tag2 threshold getting
delivered without any subject header or XSPAM
headers.


To confirm: You can follow the message in amavisd.log and you are seeing the likes of...

date etc: (05066-02) SPAM-TAG, addresses, Yes, hits=3.766 tagged_above=-999 required=3 tests=da,de,da
and...
date etc: (05066-02) Passed SPAM, addresses, Hits: 3.766, tag=-999, tag2=3, kill=5, L/Y/Y/0

...but when the user gets the message there is no Tag?

Please post your own log message from amavisd.log for a piece of spam.

Please post output from following terminal command...

grep ^[^#[:space:]] /etc/amavisd.conf

This strips out any comments and lines with leading spaces. It will show effective configuration lines.

The only option that seems to work is the option to redirect it
to a quarantine mailbox which is a bit of a pain as spamassassin
is still in it's initial traing period.


This should only occur for messages above the KILL level, which should be set to a high enough safety margin.

Anything I'm missing? I noticed there is a
amavisd.conf.personal file also in etc. Should I be
tweaking that as well?


No. The amavisd.conf file does system-wide.

-david


Server 10.4.8

Feb 9, 2007 3:30 PM in response to David_x

Hi David:

Here is an example of a passed spam:

date, etc: /usr/bin/amavisd[27596]: (27596-08) Passed SPAM, <hrunoff@innerekreis.de> -> <example@example.com>, Hits: 6.225, tag=-999, tag2=6, kill=9, 0/Y/Y/0

The spam passes on as it should but no X-SPAM header and no subject change.

Here's the output from grep ^[^#[:space:]] /etc/amavisd.conf:

$daemon_user = 'clamav'; # (no default; customary: vscan or amavis)
$daemon_group = 'clamav'; # (no default; customary: vscan or amavis)
$TEMPBASE = $MYHOME; # (must be set if other config vars use is)
$ENV{TMPDIR} = $TEMPBASE; # wise, but usually not necessary
$max_servers = 2; # number of pre-forked children (default 2)
$max_requests = 10; # retire a child after that many accepts (default 10)
$child_timeout=5*60; # abort child if it does not complete each task in n sec
@local_domains_acl = ( ".$mydomain" ); # $mydomain and its subdomains
$unix_socketname = "$MYHOME/amavisd.sock"; # amavis helper protocol socket
$inet_socket_port = 10024; # accept SMTP on this local TCP port
@inet_acl = qw( 127.0.0.1 ); # allow SMTP access only from localhost IP
$DO_SYSLOG = 0; # (defaults to false)
$LOGFILE = "/var/log/amavis.log"; # (defaults to empty, no log)
$log_level = 2; # (defaults to 0)
$log_templ = '[? %#V |[? %#F |[?%#D|Not-Delivered|Passed]|BANNED name/type (%F)]|INFECTED (%V)], #
<%o> -> [<%R>|,][? %i ||, quarantine %i], Message-ID: %m, Hits: %c';
$final_virus_destiny = D_DISCARD; # (defaults to D_BOUNCE)
$final_banned_destiny = D_BOUNCE; # (defaults to D_BOUNCE)
$final_spam_destiny = D_DISCARD; # (defaults to D_REJECT)
$final_bad_header_destiny = D_PASS; # (defaults to D_PASS), D_BOUNCE suggested
$warnvirusrecip = 1; # (defaults to false (undef))
$viruses_that_fake_sender_re = new_RE(
$mailfrom_notify_admin = "virusalert\@$mydomain";
$mailfrom_notify_recip = "virusalert\@$mydomain";
$mailfrom_notify_spamadmin = "spam.police\@$mydomain";
$mailfrom_to_quarantine = undef; # original sender if undef, or set explicitly
$QUARANTINEDIR = '/var/virusmails';
$X_HEADER_TAG = 'X-Virus-Scanned'; # (default: undef)
$X_HEADER_LINE = "by amavisd-new at $mydomain";
$remove_existing_x_scanned_headers = 0; # leave existing X-Virus-Scanned alone
$remove_existing_spam_headers = 0; # leave existing X-Spam* headers alone
$keep_decoded_original_re = new_RE(
);
$banned_filename_re = new_RE(
);
$sql_select_white_black_list = undef; # undef disables SQL white/blacklisting
$recipient_delimiter = '+'; # (default is '+')
$localpart_is_case_sensitive = 0; # (default is false)
$blacklist_sender_re = new_RE(
);
map { $whitelist_sender{lc($_)}=1 } (qw(
));
$MAXLEVELS = 14; # (default is undef, no limit)
$MAXFILES = 1500; # (default is undef, no limit)
$MIN_EXPANSION_QUOTA = 100*1024; # bytes (default undef, not enforced)
$MAX_EXPANSION_QUOTA = 300*1024*1024; # bytes (default undef, not enforced)
$MIN_EXPANSION_FACTOR = 5; # times original mail size (must be specified)
$MAX_EXPANSION_FACTOR = 500; # times original mail size (must be specified)
$path = '/usr/local/sbin:/usr/local/bin:/usr/sbin:/sbin:/usr/bin:/bin';
$file = 'file'; # file(1) utility; use 3.41 or later to avoid vulnerability
$gzip = 'gzip';
$bzip2 = 'bzip2';
$lzop = 'lzop';
$uncompress = ['uncompress', 'gzip -d', 'zcat'];
$unfreeze = ['unfreeze', 'freeze -d', 'melt', 'fcat'];
$arc = ['nomarch', 'arc'];
$unarj = ['arj', 'unarj']; # both can extract, same options
$unrar = ['rar', 'unrar']; # both can extract, same options
$zoo = 'zoo';
$lha = 'lha';
$cpio = 'cpio';
$sa_local_tests_only = 1; # (default: false)
$sa_mail_body_size_limit = 64*1024; # don't waste time on SA if mail is larger
$sa_tag_level_deflt = -999; # add spam info headers if at, or above that level
$sa_tag2_level_deflt = 6.0; # add 'spam detected' headers at that level
$sa_kill_level_deflt = 9.0;
$sa_spam_subject_tag = '*****SPAM*****'; # (defaults to undef, disabled)
$sa_spam_modifies_subj = 1; # in @spam_modifies_subj_maps, default is true
@av_scanners = (
);
@av_scanners_backup = (
);
1; # insure a defined return

Thanks for looking at this.

Feb 9, 2007 4:33 PM in response to oscarthedog

Have you fixed the out-of-the-box error in Apple's distribution of SpamAssassin under 10.4.x?

sudo -s
cd /var/amavis
mv .spamassassin .spamassassin.old
su clamav
ln -s /var/clamav/.spamassassin /var/amavis/.spamassassin

When you've done this, do 'ls -la' and you should see a line like this:
lrwxr-xr-x 1 clamav clamav 25 Feb 8 21:51 .spamassassin -> /var/clamav/.spamassassin

Also run this command:
sudo su - clamav -c "sa-learn --dump magic"
Results MUST be more then 200 on both the following lines. Spamassassin will not kick-in till it has 200 HAM & SPAM to work with.
0.000 0 15258 0 non-token data: nspam
0.000 0 1935 0 non-token data: nham

Download mailbfr & spamtrainer from here:
http://osx.topicdesk.com/downloads/

Install it and issue mailbfr -h, spamtrainer -h for what commands are available. These are great tool. spamtrainer -f fixes the above out-of-box problem automatically



Mac Pro 2 x 2.66Ghz intel 2006 Mac OS X (10.4.8) ATTO SCSI320 2CH, Highpoint 2322 SATA 2 RAID

Mac Pro 2 x 2.66Ghz intel 2006 Mac OS X (10.4.8) ATTO SCSI320 2CH, Highpoint 2322 SATA 2 RAID

Feb 10, 2007 3:23 AM in response to oscarthedog

There is one config line missing...

$mydomain = 'yourdomain.com';

where "yourdomain.com" is the domain the mail server is set for. Is this genuinely missing or did you remove it from the output? (It should have appeared as the first line so maybe accidental). It should match the domain as inserted in the Mail Server settings: Domain Name box.

I'm not sure if that would cause your problem but this certainly would...

In your ServerAdmin mail settings-> Advanced-> Hosting, do you have any domains in the Virtual Hosts box? I.e., you are using virtual domains rather than local host aliases?

If you do, then you will not get the spam tag headers on any mail to these domains until you add (or uncomment) the following line in amavisd.con...

Just under the existing line... @localdomainsacl = ( ".$mydomain" );
add...

@localdomainsmaps = ( 1 );

Stop & start mail services after any changes (preferably in Terminal using: 'serveradmin stop mail' and 'serveradmin start mail'.)

If neither of the above apply, then can you post current contents of your /etc/mail/spamassassin/local.cf file again?

-david

This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

Deliver Spam with subject heading and headers not working

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.