Anti-Spam Tool League Table

About the Anti-Spam Tool League Table

To understand the data presented in these tables read Understanding Spam Filter Accuracy. Briefly:

	The Spam Hit Rate is the fraction of spams correctly identified
	The Ham Strike Rate is the fraction of ham messages incorrectly identified

Both numbers are expressed as a decimal number to 4 places. For example, a Spam Hit Rate of .9911 means the filter successfully handled 99.11% of spams; a Ham Strike Rate of .0090 means that 0.9% of hams were incorrectly identified as spam.

Sorted by Spam Hit Rate

Click to expand/hide data table.

ID	Tool	Spam Hit Rate	Ham Strike Rate	Review
0	Panda GateDefender 8200	.9954	.0000	4
40	Barracuda Networks 400	.9920	.0010	13
66	K9	.9920	.0429	14
60	Symantec Mail Security 8240	.9890	.0000	13
1	CipherTrust IronMail 345	.9880	.0000	4
31	Audiotrieve InBoxer	.9870	.0300	5
41	Proofpoint P600	.9830	.0010	8
54	SingleFin Spam Filter	.9821	.2476	8
24	Cloudmark SpamNet	.9820	.0160	5
32	AntiSpam Mail Filter	.9815	.0320	3
48	Katharion Anti-spam	.9813	.1333	8
59	Bluecat Meridius 1000	.9780	.0010	13
2	Symantec Brightmail Anti-spam 6.0	.9769	.0000	11
20	CRM-114	.9756	.0089	1
45	Vircom ModusGate 3.0	.9723	.1238	8
3	Sybari Advanced Spam Defense	.9701	.0000	9
69	Windows Mail 'high'	.9690	.1320	14
21	CRM-114	.9684	.0120	3
22	Sophos PureMessage 4.6	.9668	.0126	11
11	SpamProbe	.9657	.0014	3
17	TrendMicro InterScan Suite 2.8	.9654	.0060	11
29	IronPort C100	.9630	.0000	13
27	iHateSpam 4	.9620	.0240	5
55	Tumbleweed MailGate 3.0	.9620	.0020	13
12	SpamAssassin	.9610	.0014	3
15	DSPAM	.9599	.0031	3
28	Bayespam	.9594	.0281	3
14	Bayesian Mail Filter	.9589	.0031	3
61	KMail/SA	.9580	.0000	14
58	Clearswift Mailsweeper for SMTP 4.3 with ASF	.9574	.4190	8
4	Mirapoint Message Director	.9566	.0000	11
10	SpamBayes	.9562	.0014	3
34	Greenview Data SpamStopsHere	.9544	.0381	8
53	MXLogic Spam Blocking	.9544	.2238	8
62	Apple Mail	.9510	.0000	14
64	Thunderbird 1.5.0.8	.9500	.0026	14
65	Thunderbird 2.0, 0.B1	.9500	.0026	14
68	Spamihilator	.9500	.0530	14
42	Espion Interceptor 1.3.25	.9484	.0857	8
30	BorderWare Mxtreme	.9342	.0286	8
23	iFile	.9320	.0152	3
46	Postini Perimeter Manager	.9230	.1286	8
16	Quick Spam Filter	.9157	.0039	3
13	DigitalInfo MailPoint 3000	.9142	.0023	11
7	DSPAM	.9116	.0004	1
36	Aladdin System Spamcatcher	.9110	.0429	5
67	Windows Mail 'low'	.9110	.0440	14
43	MS Exchange IMF	.9070	.1100	2
25	FrontBridge TrueProtect	.9042	.0170	11
63	Outlook 2007	.9040	.0018	14
9	bogofilter	.9030	.0011	3
18	RoaringPenguin CanIt-PRO 2.0b	.9006	.0072	11
19	RoaringPenguin CanIt-PRO 2.0b	.9000	.0082	10
8	Annoyance Filter	.8898	.0007	3
50	SpamAssassin	.8800	.1477	10
52	Erado Total Control	.8662	.1666	8
39	Trend Micro PC-cillin	.8630	.0690	5
5	BorderWare MXtreme MX-200	.8607	.0000	11
6	SpamOracle	.8371	.0000	3
51	McAfee SpamKiller 6.0	.8000	.1500	6
37	Norton Antispam 2004	.7950	.0520	5
56	SurfControl E-mail Filter 4.7	.7877	.2524	8
57	Panda Software Platinum	.7530	.3700	5
44	JspamFilter v3.5	.7504	.1143	8
35	McAfee SpamKiller 6.0	.7500	.0400	7
49	WebWasher SPAMEquator Prime	.7190	.1381	8
33	RoaringPenguin CanIt-PRO	.6472	.0333	8
47	Lyris Tech. MailShield 3.1	.6270	.1300	5
38	Symantec AntiVirus for SMTP	.4851	.0524	8
26	Paessler NoSpamToday	.3176	.0190	8

Sorted by Ham Strike Rate

Click to expand/hide data table.

ID	Tool	Spam Hit Rate	Ham Strike Rate	Review
0	Panda GateDefender 8200	.9954	.0000	4
1	CipherTrust IronMail 345	.9880	.0000	4
2	Symantec Brightmail Anti-spam 6.0	.9769	.0000	11
3	Sybari Advanced Spam Defense	.9701	.0000	9
4	Mirapoint Message Director	.9566	.0000	11
5	BorderWare MXtreme MX-200	.8607	.0000	11
6	SpamOracle	.8371	.0000	3
29	IronPort C100	.9630	.0000	13
60	Symantec Mail Security 8240	.9890	.0000	13
61	KMail/SA	.9580	.0000	14
62	Apple Mail	.9510	.0000	14
7	DSPAM	.9116	.0004	1
8	Annoyance Filter	.8898	.0007	3
40	Barracuda Networks 400	.9920	.0010	13
41	Proofpoint P600	.9830	.0010	8
59	Bluecat Meridius 1000	.9780	.0010	13
9	bogofilter	.9030	.0011	3
10	SpamBayes	.9562	.0014	3
11	SpamProbe	.9657	.0014	3
12	SpamAssassin	.9610	.0014	3
63	Outlook 2007	.9040	.0018	14
55	Tumbleweed MailGate 3.0	.9620	.0020	13
13	DigitalInfo MailPoint 3000	.9142	.0023	11
64	Thunderbird 1.5.0.8	.9500	.0026	14
65	Thunderbird 2.0, 0.B1	.9500	.0026	14
14	Bayesian Mail Filter	.9589	.0031	3
15	DSPAM	.9599	.0031	3
16	Quick Spam Filter	.9157	.0039	3
17	TrendMicro InterScan Suite 2.8	.9654	.0060	11
18	RoaringPenguin CanIt-PRO 2.0b	.9006	.0072	11
19	RoaringPenguin CanIt-PRO 2.0b	.9000	.0082	10
20	CRM-114	.9756	.0089	1
21	CRM-114	.9684	.0120	3
22	Sophos PureMessage 4.6	.9668	.0126	11
23	iFile	.9320	.0152	3
24	Cloudmark SpamNet	.9820	.0160	5
25	FrontBridge TrueProtect	.9042	.0170	11
26	Paessler NoSpamToday	.3176	.0190	8
27	iHateSpam 4	.9620	.0240	5
28	Bayespam	.9594	.0281	3
30	BorderWare Mxtreme	.9342	.0286	8
31	Audiotrieve InBoxer	.9870	.0300	5
32	AntiSpam Mail Filter	.9815	.0320	3
33	RoaringPenguin CanIt-PRO	.6472	.0333	8
34	Greenview Data SpamStopsHere	.9544	.0381	8
35	McAfee SpamKiller 6.0	.7500	.0400	7
36	Aladdin System Spamcatcher	.9110	.0429	5
66	K9	.9920	.0429	14
67	Windows Mail 'low'	.9110	.0440	14
37	Norton Antispam 2004	.7950	.0520	5
38	Symantec AntiVirus for SMTP	.4851	.0524	8
68	Spamihilator	.9500	.0530	14
39	Trend Micro PC-cillin	.8630	.0690	5
42	Espion Interceptor 1.3.25	.9484	.0857	8
43	MS Exchange IMF	.9070	.1100	2
44	JspamFilter v3.5	.7504	.1143	8
45	Vircom ModusGate 3.0	.9723	.1238	8
46	Postini Perimeter Manager	.9230	.1286	8
47	Lyris Tech. MailShield 3.1	.6270	.1300	5
69	Windows Mail 'high'	.9690	.1320	14
48	Katharion Anti-spam	.9813	.1333	8
49	WebWasher SPAMEquator Prime	.7190	.1381	8
50	SpamAssassin	.8800	.1477	10
51	McAfee SpamKiller 6.0	.8000	.1500	6
52	Erado Total Control	.8662	.1666	8
53	MXLogic Spam Blocking	.9544	.2238	8
54	SingleFin Spam Filter	.9821	.2476	8
56	SurfControl E-mail Filter 4.7	.7877	.2524	8
57	Panda Software Platinum	.7530	.3700	5
58	Clearswift Mailsweeper for SMTP 4.3 with ASF	.9574	.4190	8

Sorted by Review

Click to expand/hide data table.

ID	Tool	Spam Hit Rate	Ham Strike Rate	Review
7	DSPAM	.9116	.0004	1
20	CRM-114	.9756	.0089	1
43	MS Exchange IMF	.9070	.1100	2
6	SpamOracle	.8371	.0000	3
8	Annoyance Filter	.8898	.0007	3
9	bogofilter	.9030	.0011	3
10	SpamBayes	.9562	.0014	3
11	SpamProbe	.9657	.0014	3
12	SpamAssassin	.9610	.0014	3
14	Bayesian Mail Filter	.9589	.0031	3
15	DSPAM	.9599	.0031	3
16	Quick Spam Filter	.9157	.0039	3
21	CRM-114	.9684	.0120	3
23	iFile	.9320	.0152	3
28	Bayespam	.9594	.0281	3
32	AntiSpam Mail Filter	.9815	.0320	3
0	Panda GateDefender 8200	.9954	.0000	4
1	CipherTrust IronMail 345	.9880	.0000	4
24	Cloudmark SpamNet	.9820	.0160	5
27	iHateSpam 4	.9620	.0240	5
31	Audiotrieve InBoxer	.9870	.0300	5
36	Aladdin System Spamcatcher	.9110	.0429	5
37	Norton Antispam 2004	.7950	.0520	5
39	Trend Micro PC-cillin	.8630	.0690	5
47	Lyris Tech. MailShield 3.1	.6270	.1300	5
57	Panda Software Platinum	.7530	.3700	5
51	McAfee SpamKiller 6.0	.8000	.1500	6
35	McAfee SpamKiller 6.0	.7500	.0400	7
26	Paessler NoSpamToday	.3176	.0190	8
30	BorderWare Mxtreme	.9342	.0286	8
33	RoaringPenguin CanIt-PRO	.6472	.0333	8
34	Greenview Data SpamStopsHere	.9544	.0381	8
38	Symantec AntiVirus for SMTP	.4851	.0524	8
41	Proofpoint P600	.9830	.0010	8
42	Espion Interceptor 1.3.25	.9484	.0857	8
44	JspamFilter v3.5	.7504	.1143	8
45	Vircom ModusGate 3.0	.9723	.1238	8
46	Postini Perimeter Manager	.9230	.1286	8
48	Katharion Anti-spam	.9813	.1333	8
49	WebWasher SPAMEquator Prime	.7190	.1381	8
52	Erado Total Control	.8662	.1666	8
53	MXLogic Spam Blocking	.9544	.2238	8
54	SingleFin Spam Filter	.9821	.2476	8
56	SurfControl E-mail Filter 4.7	.7877	.2524	8
58	Clearswift Mailsweeper for SMTP 4.3 with ASF	.9574	.4190	8
3	Sybari Advanced Spam Defense	.9701	.0000	9
19	RoaringPenguin CanIt-PRO 2.0b	.9000	.0082	10
50	SpamAssassin	.8800	.1477	10
2	Symantec Brightmail Anti-spam 6.0	.9769	.0000	11
4	Mirapoint Message Director	.9566	.0000	11
5	BorderWare MXtreme MX-200	.8607	.0000	11
13	DigitalInfo MailPoint 3000	.9142	.0023	11
17	TrendMicro InterScan Suite 2.8	.9654	.0060	11
18	RoaringPenguin CanIt-PRO 2.0b	.9006	.0072	11
22	Sophos PureMessage 4.6	.9668	.0126	11
25	FrontBridge TrueProtect	.9042	.0170	11
29	IronPort C100	.9630	.0000	13
40	Barracuda Networks 400	.9920	.0010	13
55	Tumbleweed MailGate 3.0	.9620	.0020	13
59	Bluecat Meridius 1000	.9780	.0010	13
60	Symantec Mail Security 8240	.9890	.0000	13
61	KMail/SA	.9580	.0000	14
62	Apple Mail	.9510	.0000	14
63	Outlook 2007	.9040	.0018	14
64	Thunderbird 1.5.0.8	.9500	.0026	14
65	Thunderbird 2.0, 0.B1	.9500	.0026	14
66	K9	.9920	.0429	14
67	Windows Mail 'low'	.9110	.0440	14
68	Spamihilator	.9500	.0530	14
69	Windows Mail 'high'	.9690	.1320	14

FAQ

Where's POPFile?

All the data in the league table comes from published reviews of anti-spam software. I don't know of a published review of POPFile that contains enough data to calculate the Spam Hit Rate and Ham Strike Rate. I'm not going to add my own figures for POPFile because that would be biased. If you know of a review that covers POPFile please email it to me.

As a chart

The y-axis shows the Spam Hit Rate from .3000 to 1; the x-axis shows the Ham Strike Rate from .5000 to 0 (note that the x-axis is in descending order left to right). The best spam filters appear in the top-right hand corner.

Hover on any cross to find the filter name; click any cross to read the associated review.

As a chart (only "good" filters)

The y-axis shows the Spam Hit Rate from .9000 to 1; the x-axis shows the Ham Strike Rate from .0100 to 0 (note that the x-axis is in descending order left to right). This chart only shows spam filters that have good performance (i.e. filter greater than .9000 of spam and lose less than .0100 of ham).

Hover on any cross to find the filter name; click any cross to read the associated review.