To SPF or not to SPF

Since many people keep asking about SPF, I thought I'd publish a few facts gathered from my own experience on servers managed by myself. These are just facts and some personal thoughts. I will not get into any "evangelistic" discussions about SPF.

Although to me it was clear from the beginning that SPF is not a solution I thought I'd give it a try to gather some real world data. In order to be able to analyse the information better, I used a server that has a decent amount of load, but is not exaggeratedly busy.

Basic components used:
Mac OS X 10.4.8 with latest security updates.
Postfix 2.1.5
amavisd 2.4.4
SpamAssassin 3.1.7
ClamAV 0.88.7 with clamd instead of clamscan

SPF related components used:
Mail::SPF and Mail::SPF::Query (tried both in seperate moments)
postfix-policyd-spf-perl (a perl script that calls above PERL script(s) and is used together with postfix' policy server)

I modified postfix-policyd-spf-perl so that it only logs it's "findings", but never actually blocks mail.


Configuration:
I lowered all SPF related SpamAssassin scores to 0.001, because I just wanted to see what gets triggered, but was not interested in action being taken.

SpamAssassin has a well trained Bayes database, all necessary fixes and some additional rules (the same rules that get installed by "spamtrainer -a").

Postfix was configured to reject as much spam as possible before even calling a content filter. (Based on what I published in my document "Frontline spam defense for Mac OS X Server")

The policy server is called after all other checks are done (this is very important, otherwise you can become an open relay if a spammer has actually published an SPF record). It is also more efficient on resources.

I configured the script to log/act on following conditions:
No SPF record = none (equals pass, because rejecting mail from domains without SPF records, basically means ludicrous amounts of false positives)

Correct SPF record and mail sender matches = pass

Correct SPF record but mail sender doesn't match = fail

DNS Errors while looking up = softfail (pass)


So much for the basic information, which is much longer than the statistical data gathered, but nonetheless important for a full picture.

Now to the data:
Test Period: 2 weeks
Average number of incoming mails per day: 32'000
Average number of rejected mails per day: 28'000
(Yes almost 90% of mail is spam or viruses)

Now the juicy bit. Out of the average of 28'000 mails caught by Postfix, ClamAV and SpamAssassin, how many do you think would NOT have been caught without SPF checking in place?:
100 or roughly 0.35%

I got "SPF fail" on an average of 208 mails. 108 of them had been caught by the other measures in place already.

So is this worth the effort and the extra hit on server and network resources? Hardly. At least not in my book.


Remarks:

Why is SPF praised, yet at the same time so inefficent?

The main problem is that it relies on domain owners publishing a voluntary SPF TXT DNS record. This record declares which servers may or may not send mail for said domain. It can be defined as a "strict" record (listing allowed machines and disallowing any other server) or as a "loose" record (listing allowed machines, but not excluding that other servers may send mail for the domain). Most published SPF records are not strict (google.com for example). Why? Because a domain's user on the move forced to use a different ISP's mail server would not be able to send mail anymore (yes I know you can always have different submission ports to prevent being blocked from accessing your own server, but not everybody does). Now when an SPF record is not "strict", rejecting mail not coming from designated servers, yields a very high risk of false positives and thus, most sysadmins will let it through (maybe with an increased SpamAssassin score, but that's about it).

Another problem is that an enormous amount of domains do not publish an SPF record at all (mac.com for example doesn't). Do you want to block those mails? Probably not.


Should I use SPF checking?

This is something you will have to decide for yourself. I am definitely not using it. On my servers it has proven to be an ineffective method of catching spam. It eats cpu cycles and does network roundtrips without a real benefit. It's not that the perl scripts and dns lookups bring my system to its knees, far from it, but I have a strict policy on wasting resources.


Should I publish an SPF record for my domain(s)?

Again, you will have to decide for yourself. Under normal circumstances a mail server will let your mail through even if you do not publish an SPF record. There are however some inept system administrators that do block mail from domains without an SPF record). In many cases, not having an SPF record is safer than having a strict one. I have decided to publish an SPF record for my domains. This doesn't mean I believe in it. It's just something that doesn't cost me any extra resources. Plus I have full control over my servers and do not rely on any ISP doing configuration work for me.


I didn't get into the technical tidbits about SPF's reliabilty, spoofing and so on. There is enough information on the internet on this. All I wanted was to illustrate a real world situation.

Feel free to post questions.

Alex