It’s been a virtually spam-free week here in my new home on the web. And it’s not because the spammers are taking a holiday break. My Qmail logs report that, since the shields went up on Monday morning, over 600 chunks of spam have been deflected. What’s the secret? Nothing new and innovative, just an idea whose time has definitely come: real-time source blacklisting. Simple, efficient, and effective.
A New York Times article about the new anti-spam legislation (which some people think will make the problem worse instead of better) mentioned that 90% of spam comes from 200 spammers. So, just rejecting any mail from those 200 spammers seems more sensible than scanning all incoming mail for clues that it might be spam.
Of course, “experts” hasten to point out that it’s not that simple. You can’t tell if mail is from a spammer because they hide their identity, forge headers, route mail through foreign servers, yada yada yada. And all that is true. But there’s one thing they can’t hide: the address of the “final hop”, the server that hands off the spam to my server. There’s no way a mail server can deliver a message without disclosing its IP address. And that address is a far more accurate indicator of whether a message is valuable or spam than all the filters that look for disguised porn ads.
As the legitimate ISPs are getting aggressive about enforcing their anti-spam terms of service, and security holes that allow server hijacking are being closed, the spammers’ choice of routing has become very predictable. The vast majority of spam is now coming through servers in Asia or South America, or hacked broadband customers. For quite a while, I’ve been reporting most of my spam to Spam Cop, a service which scans the mail and sends abuse report to the administrators of the sending server, and to the hosts of any websites advertised in the spam. It’s really frustrating to see all the abuse reports going to Comcast, ATT, RoadRunner, etc. I’m sitting here, unable to get broadband service, maxing out my flaky dialup connection downloading spam spewed by clueless Comcast customers running hacked Winblows systems, bragging about how fast they can download pirated movies while not knowing that their machines are flooding the world with spam. There really ought to be a requirement for an IQ test before getting a broadband connection, to protect the net from these idiots. Well, at least now I’m protected; spam from those idiots ain’t getting into my system.
So how does it all work? How do I know who to block? It’s elegant and simple. There are a bunch of blacklist servers which provide IP addresses that are “spam-friendly”. The criteria for being listed vary among the different lists, but they include such things as being reported as a source of spam, being an open relay susceptible to hijacking, being a dynamic address or broadband client which has no business running an SMTP server, etc. The lists are constantly being updated, with new hosts being added as they are detected to be problematic, and others being removed if they clean up their act. Some blacklists are more aggressive than others, offering stronger protection with a slightly higher risk of blocking legitimate mail, and some are more generous than others about removing repenters. Any mail server can query one or more of the blacklist servers whenever a delivery is attempted, and refuse to accept the message if the sending server is blacklisted. Shield strength can be adjusted by choosing which list(s) to query. Since the blacklist servers are queried in real-time for each message, the most current list is always used.
I’m currently blocking mail from sources blacklisted by DSBL, CBL, SpamHaus, SORBS, and NJABL. This combination is doing a pretty good job of stopping spam. I’ve had a few unwanted messages that slipped through in the last week, but less than a dozen, far fewer than before. I really have no way of being absolutely sure that I haven’t blocked a message that I really wanted, but it’s really doubtful that anybody I want to hear from would be routing mail through any of the blacklisted hosts.
Blacklisting, in spite of a name that is vaguely reminiscent of McCarthyism, really makes sense on many levels. It was somewhat controversial when it first started several years ago. But the spam problem has gotten so bad that many cures which might have been controversial at one time now seem like no-brainers. And the major objection to blacklisting, that it can block a lot of legitimate mail if a “good” server gets blacklisted for some reason, is far less of an issue as responsible providers are tightening up their systems to avoid abuse. Back when the net was a kinder, gentler place, open relays were commonplace, just waiting for exploitation by spammers. In fact, my employer’s primary outbound SMTP server was blacklisted for a while a few years ago after spammers realized it was open and relayed thousands of messages through it. For a while, it was impossible to send mail from our campus to any site that was blacklisting. At that time, being blacklisted was less of a problem than it is today, because fewer sites were using the blacklists. But we realized we had an obligation, to our users and to the entire net, to eliminate the opportunities for abuse and get ourselves off the blacklists. That’s part of the overall cycle that is making blacklisting a more viable solution. As the servers that are responsible for legitimate mail take the steps necessary to get off the lists, blacklisting becomes less likely to block good mail. And, as that becomes less likely, more sites start to use it. And, as more sites start to use it, more sites realize they need to stay off the lists.
From a technological point of view, it’s far more efficient to simply refuse to accept messages from suspicious hosts, than to accept all incoming messages and then scan them checking for telltale signs of spam, and hope I’m not throwing away messages from friends sending me Viagra jokes. Inbound servers aren’t clogged with a lot of messages that are going to be scanned by filters and then thrown away. Net traffic decreases as the spam is rejected prior to delivery.
On a philosophical level, it provides the sort of voluntary, collaborative, self-policing that the Net needs. There’s no hamfisted, overzealous, government intervention. Nobody is forcing anything on anybody else. As the blacklist providers point out, they’re not blocking anybody’s mail. They’re simply providing lists for people to use as they choose. Those who wish to participate in a spam-free net can use the lists, and make sure they don’t commit offenses that would get their own servers listed. Those who don’t like the idea of limiting spam are free to spam their hearts out, but they are going to find that the number of servers that accept their messages is steadily decreasing. The quality of the blacklists is “market-driven”. Those lists that either block legitimate mail, or fail to block a lot of spam-friendly hosts, won’t be used by many servers. Those that provide a reliable way to identify spam senders before accepting mail from them will continue to keep my mailbox spam-free.