April 21st, 2008
Posted in Uncategorized
Blocking Spam In 2008
Like a shepherd, the duty of a bot herder (botnet operator) is to keep his/her botnet army intact. Bot herders make money by amassing a botnet, then contracting out the botnet services to spammers. That’s right, spammers employ bot herders to do the dirty work for them!
Bot herders only get paid by the spammer when a message is actually delivered to the receiving email server. For those readers familiar with SMTP protocol, this means that the bot herder only gets paid once the server has sent 250 Ok after the DATA phase. In order to make a lot of money, bot herders have to send as much as possible in the shortest possible time. If a zombie is being blocked, the bot herder doesn’t make any money. The bot herder only makes money when a message is actually received by the receiving email server.
Spamming software is impatient. In programming terms, spamming software has a very low timeout. The SMTP RFC recommends that email servers wait at least three minutes for each chunk of data they send to be received by the receiving server and acknowledged via a TCP acknowledgement packet. Furthermore, the RFC recommends that senders wait at least ten minutes for the final message delivery acknowledgement.
These long timeouts were established because in the early days of the Internet, the infrastructure was slow and unreliable, and the machines were easily overloaded, leading to frequent message delivery delays. Today, email servers and our networks are much faster at processing incoming messages in a matter of seconds. Delays still occur, but the
timeouts defined in the RFC are vastly higher than what is required in today’s world.
Because bot herders don’t get paid until they receive the 250 Ok, their software earns a higher profit by disconnecting after a few seconds and seeking out new victims whose servers respond more quickly. Bots can’t afford to wait for a slow connection to go through, and they can’t risk being discovered and put on a blacklist.
A few years ago, the MIT Spam Conference was a very interesting place. Each year, bright-eyed graduate students and intrepid industry types would present new filtering techniques that pushed the accuracy of spam filters to new levels. For the past three years, improvements in spam filter effectiveness has plateaued. A great result is a paper that shows the accuracy improvement of half a percent. Spam filtering has essentially become maxed out as a technology, and there isn’t much more we can do but tweak rules to avoid falling behind the spammer’s arms race.
Similarly, reputation systems which identify suspicious IP addresses have become asymptotic in their effectiveness. The spread of botnets has led to a virtually inexhaustible supply of new IP addresses, that spam us a few times and then disappear forever. Most of the large anti-spam companies now have comprehensive blacklists that are updated every minute.
In other words, anti-spam systems worldwide are blocking everything they possibly can. And yet spam continues to grow as a problem — it’s unbelievable. So what can we do?
Bill Gates was right in 2004. He boldly posited that the way to solve the spam problem was to introduce a cost barrier that caused spamming to be no longer profitable. Unfortunately, spammers created botnets, which have rendered to them more computing power than most governments. One way to think of the problem is that the spammers have millions of computers. You only have a handful. And you have to pay for yours. Who’s going to win? While we can’t win the spam war with better filters or better blacklists, there are alternatives.
To deter spamming we must undermine spammers, not simply block messages. You can make botnets unprofitable by slowing down SMTP traffic from spammers. This not only gives the receiver control of each email connection, but it also consumes sender resources to reduce the spammer’s sending rate significantly.
Imagine the chaos at an airport without air traffic controllers and you begin to see why mail servers need email traffic control.
NEXT: Post #7 Slowing Things Down
PREVIOUS: Post #5 Why Are Botnets So Difficult To Stop?
Tags: anti-spam, bill gates, bot herders, data phase, profit, RFC, smtp, spam, spammers, trickle blog
April 7th, 2008
Posted in Uncategorized
Once Promising Proposals for a Final Ultimate Solution to the Spam Problem (FUSSP)
“Two years from now, spam will be solved.”
That was Bill Gates’ famous pronouncement back in 2004. Microsoft, Yahoo and the open source community devised two techniques that they believed would eradicate spam. The first was sender authentication, which allowed email senders to provide a list of the servers permitted to send email for users within their domain. The idea was that sender authentication would eliminate spammers spoofing legitimate email addresses, and allow for the creation of a permanent, ironclad white list of trustworthy domains that never send spam, thus allowing recipients to simply block everything not on the white list and end spam forever.
Another idea pitched in 2004 was the computational challenge. Senders would, upon connecting to a receiving email server, have to spend considerable CPU cycles computing the answer to a mathematical challenge provided by the receiving server. Bill Gates believed this approach would stop spam by making it cost too much to send the high volumes of email required to make spamming profitable.
Unfortunately, neither sender authentication nor the computational challenge technique resolved the spam problem. Computational challenges were rejected as being too costly for legitimate bulk email senders (airlines, banks, open source mailing lists, etc.) And sender authentication while eventually enjoying wide-spread adoption in the form of DKIM and SenderID, proved prone to errors. As as result it has remained useful mostly for the acceptance of legitimate email and phishing protection rather than the rejection of spam.
By 2005, what the anti-spam community was getting right was content filtering. When spam filters had reached above the 90 per cent accuracy level, spam transitioned from a problem of content to a problem of volume, the spammers simply send more spam. And they can do this because the recipient pays the cost of content filtering rather than the spammer.
The cost of a resource-consuming filtering system increases during high traffic loads. If you block spam content, spammers will find new ways to get around it. Bill Gates was right, the only way to stop them is to create difficulty by making spam too costly to send. If you do spammers are left to find new targets that are easier to hit.
NEXT: Post #4 Spamonomics: The Economics of Spamming
PREVIOUS: Post #2 Prohibition Induces “Botlegging”
Tags: accuracy, anti-spam, bill gates, content-filters, dkim, economics, high traffic loads, microsoft, spam, spammers, spamonomics, yahoo
April 3rd, 2008
Posted in Uncategorized
Prohibition Induces “Botlegging”
Spamming is a “tragedy of the commons,” in which a finite resource (our time and attention) is abused at low cost by a minority (the spammers). Like many such tragedies in our human history, prohibition has been seen as the quick fix. Classic targets of prohibitionism include alcohol, drugs, and gambling. The idea is simple really. Stop spammers from profiting by making the actions illegal, enforceable and a harmful choice to the culprit. However, this kind of law is difficult to enforce.
In 2003, American legislators passed the CAN-SPAM Act (Controlling the Assault of Non-Solicited Pornography And Marketing). CAN-SPAM made it illegal to send unsolicited bulk email with a deceiving subject line and forced legitimate senders to identity themselves with a full mailing address.
So why then, does spam volume continue to rise despite an increased adoption of spam blocking mechanisms worldwide?
Several years have passed and spam volume is higher than ever. While CAN-SPAM is rightly criticized for not ending the spam problem, its most significant side effect was to force spamming underground and out of the reach of law enforcement. Face with service interruptions, spammers began in early 2004 to migrate their operations to a highly scalable distribution platform immune to law enforcement: the botnet.
By the end of the same year, the majority of spam was being delivered by decentralized networks such as “Phatbot” – and nowadays by Storm, Mega-D, and Srizbi – lending little hope to Bill Gates’ famous pronouncement that spam would be beaten before the end of 2006.
The fact is that there are limitations with each anti-spam technique. Content filters are a core component of that architecture and are very effective at separate spam from email once they receive and recognize it. DNSBLs can block bad senders from known IP addresses once they known the sender is bad. But what happens when a botnet harvests new zombies with IP addresses unknown to DNSBLs and uses those to send new spam campaigns – something that happens every day? Discarding spam after you receive it does nothing to decrease high spam traffic from new campaigns. What is needed is a combination of the best-of-breed elements suited to deal with each type of spam: known content, unknown content, known senders and most importantly the unknown sender.
If you’re doubling servers to deal with heavy spam loads, your infrastructure costs are under control of the spammers who can just keep sending more spam. What you need is a new solution that can block most spam without having to receive the message first in order to get the costs and the load back under control and ensure your infrastructure is used to deliver legitimate mail first.
NEXT: Post #3 Once Promising Proposals for a Final Ultimate Solution to the Spam Problem (FUSSP)
PREVIOUS: Post #1 Short History on Spam Protection
Tags: anti-spam, bill gates, botnet, CAN-SPAM, dnsbl, mega-d, spam, storm, trickle blog
January 31st, 2008
Posted in Uncategorized
This morning I checked an old Hotmail account and I was surprised to see an e-mail from Bill Gates. The From header was “Bill Gates (billgates_2008lottery@yahoo.fr)” which was a little surprising. I would have thought that Bill would be sending it from a Microsoft or at least a Hotmail address and not a competitor such as Yahoo. I put that thought at the back of my mind since I was excited to see why he was e-mailing me. Here’s the message I received:
The content of the message was in French. I put this down to the fact that Microsoft is based in Redmond, Washington which borders us here in Canada and perhaps some French Canadian influence had started to spread across the border in the USA?
After reading the e-mail, I discovered I was a winner of the “BILL GATES FOUNDATION LOTTERY FOR INTERNET EXPANSION IN AFRICA”. I realized this was quite a big deal since capitalization was used in the name! All I have to do is contact Claude Verges at the “law firm” and provide my name, address, phone, fax, e-mail and a copy of my national identity card or passport. All of which would be very useful for identity theft so I think I’ll pass on this occasions. Sorry Bill!
Tags: bill gates, hotmail, microsoft
