paul's anti-spam evolutions

I get about 150-200 non-spam emails a day, and send about 60-80. I receive hundreds of spams a day, or thousands when I enable my test magnet. Here are the anti-spam techniques I've evolved over the past few years.

First generation (2001)

A surprisingly effective simple script written in procmail:

1. Email from friends: send to inbox
2. Email from "evil" strangers (content match bad words list): delete!
3. Email from strangers directly to me: hold for daily review
4. Else hold in a "bulk" folder for weekly review

Second generation (2002-2003)

Design of the family-friendly SpamSubtract which improved the above algorithm and added a super efficient, separate mail review window. No spam ever touched my real email client, for improved performance, safer anti-virus, and less interruption.

Third Generation (2004)

The new "V3" system is designed for Microsoft Outlook used with a Unix mail server, and its integration between desktop and server will provide notable improvements over any desktop-only or server-only anti-spam solution. V3 will also work well with real-time "low bandwidth" email devices such as the Blackberry. Here are the V3 components:

1. Super Friends. A simple OutlookFTP utility keeps all my Outlook contacts (cf Sperry) on my Linux server. If I send email to Joseph Mahoney at jm@example.com, the mail server can then immediately allow email from "Joseph Mahoney" from any of his email addresses, and allow email from any of Joe's colleagues at "example.com". The system has a small exception list of domains (aol.com etc) and names to not treat like this. Email from friends marked high priority will cause my Blackberry to buzz on receipt. All friend emails are also fed to my Bayesian system as known-good-content. This is important, since if Joe sends me email about flying pigs, I care about flying pigs.

2. Stranger Analysis. All other email is marked as low priority, passed optionally through a Bayesian system (e.g., SpamProbe) for analysis, and is then deleted (for high scores), or forwarded into strangers file for later review. (Soon: strip attachments, mask bad words missed by Bayesian, put images one click away.)

3. Challenge response (optional). Create a minimally invasive challenge response system, working hard (by above rules) to never challenge a good guy, nor an obvious bad guy.

4. Outlook Override (optional) will allow me to one-click set any email to "Friend" or "Spam", immediately correcting the rules on my server.

5. Hide New. All my published email addresses (e.g., . ) will use safemailto, to make my mailto links usable by humans but hidden to spam robots. My domain name registration email will be filtered specially. All email addresses I give to any company will be coded so I can tell when my email address gets sold by whom. (See also.)

6. Bounce Old (optional). All my old email addresses which have been spam-harvested but which could still possibly get some real email will be handled by my bouncer script. This will send a generic bounce message to the sender. E.g., try paul@bostonlight.org.

Back to Paul's spam page.