Fifty questions for Spamhaus, with our answers
Ken Magill, who writes a weekly marketing blog called “The Magill Report”, recently solicited his readers to submit questions for Steve Linford, principle at Spamhaus. Ken’s readers sent in 51 questions, and I thought it would be fun to take a crack at answering them. Here goes:
1. How can Spamhaus work directly with legitimate marketers when issues arise? Wouldn’t it best serve customers and the overall email industry to resolve issues in good faith (as opposed to staying at arms’ length)?
Spamhaus already works directly with marketers – at least, those marketers who are reputable enough to attend conferences like M3AAWG. Spamhaus contributes very actively in such forums, and in a constructive way to help marketers understand how to behave in such a way that they won’t qualify for a listing.
2. As more retailers offer to “email your receipt” in stores, the problem of miss-typed email addresses is likely to increase, and hitting Spamhaus traps will be more prevalent. Is there some way for Spamhaus to “ignore” emails that it gets from retailers when they see a capture event type (like a receipt)? Could they eventually focus instead on ensuring that marketers have good list hygiene by ensuring that the email is no longer mailed 12 months after not activating? Or what would they recommend?
I don’t think Spamhaus has a problem with stores sending out the odd receipt to an incorrect address. The problem was stores that then went on to send that email address marketing messages. If a customer provides their address to receive a receipt, then a receipt is really all they should get. It seems rather disingenuous for the store to assume that a bit of marketing would be acceptable.
3. How is Spamhaus working with legitimate marketers to improve list hygiene? Do they have a list of ‘best practices’ that they’d ideally like brands to follow that are business friendly (getting that customer email address) as well as good for business (legitimate email address)?
We would recommend applying to join M3AAWG. Short of that, read the many published documents provided by M3AAWG, which anyone can use to greatly improve their overall mailing practices.
4. Does Spamhaus use email addresses that were used to subscribe to mailing lists and then discarded? Do old Yahoo, Gmail addresses become spam traps? How old? Also are they being tracked by Spamhaus?
I don’t think any anti-spam operation worth their salt would ever disclose what types of email addresses are used as spam traps. Generally speaking, however, a good spam trap is an address that was never used for legitimate emailing – including belonging to a mailing list. It would be very poor practice to scrape addresses from old mailing lists and turn them in to traps (say, by purchasing those expired domains). The people I talk to in the industry – who run good traps – take extensive precautions to avoid using addresses that may still receive legitimate email.
5. Are Spamhaus listings [ever] based on complaints sent to them?
I would speculate that some listings are based on complaints, but that most are based on Spamhaus’ original research.
6. If hitting spamtraps is the only criterion what is the threshold?
If Spamhaus were to reveal their thresholds, then this would permit spammers to game the system by simply limiting the number of times they hit each email address in their lists. So, count on Spamhaus never revealing anything about the algorithms they use to select an IP for listing based on trap hits.
7. How is Spamhaus certifying an ESP? What is the criteria? [Steve, I have no idea what this refers to. I considered deleting it, but included it thinking you might know what he’s asking.]
To my knowledge, Spamhaus is not in the business of certifying ESPs. If you want to be certified, contact Return Path.
8. When Spamhaus created their whitelist they chose not to permit “marketing of any sort” or permit any company applying who used an ESP. Because Spamhaus is in a uniquely privileged position with their whitelist, they could have helped the email industry with a new standard of trust. Why did they choose not to do this?
If this is true, my guess is that ESPs generally have such a poor track record that it would be difficult for Spamhaus to pick and choose the very few ESPs who behave well enough to warrant being on the SWL.
9. Does Spamhaus believe that email should be delivered to consumers who have opted-in to email marketing from brands? [I know the short answer is yes, but left this one in in case you want to elaborate.]
I would say yes, with the following caveats: a) the consumer needs to know he or she is opting in to receive marketing messages, and b) the messages subsequently need to be highly correlated with what the consumer thought he or she would be receiving.
10. How can professional email marketers who wish to get opt-in emails delivered work with Spamhaus and other important providers of spam detection to help ensure spam is not delivered and other communications are? [Here again, I know the short answer is stop spamming, but I left it in anyway.]
Isn’t “professional email marketing” the art of getting stuff delivered that maybe shouldn’t get delivered?
11. What is their goal with CSS and do they feel their achieving it? Are they catching the “bad guys” so to speak or could it be acknowledged that ‘babies are being thrown out with the bathwater’? [This one’s from a reader who says they’re doing everything right and yet got caught in you anti-show-shoe spamming efforts somehow.]
The goal is clearly laid out on the Spamhaus CSS page:
As a snowshoe spreads the weight of a traveler across a wide area of snow, snowshoe spammers spread their spam output across many IPs and domains, diluting reputation metrics and evading filters. Snowshoe spammers frequently use many fictitious business names (DBAs), false names and identities, concealed anonymous domains and frequently changing postal dropboxes and voicemail drops to prevent others from connecting snowshoe spam operations to one another and recognizing who is behind the operations and the spam they send.
Spamhaus believes that the problem of snowshoe spam is now large enough to warrant a special response aimed specifically at it. The CSS is our response to this problem, and is a collaborative effort of Spamhaus and the CBL.
12. What trips a CSS listing – spamtraps?
Spamhaus mentions that they are working with the CBL, which implies that much of the detection is based on traps.
13. How real-time are the [SBL] listings? In other words, if you sent something a week ago, could that cause you a listing now, or does it happen from the most recent mail only?
My understanding of the SBL is that it’s a manually curated list, based on a huge amount of automatically collected information. I’m sure listing speed is based on whether someone is awake and ready to hit the button.
14. It’s clear from Spamhaus ‘recent SBL listings’ tracking list that the vast majority of SBLs are related to criminal behavior, most of which involves truly nefarious and malicious activity. It’s also clear from most of Spamhaus ISP ‘users’ that they no longer deliver most ‘spam’ or even ‘bacn’ to the Inbox and their filters are highly customized to identify unwanted messaging from dedicated IP address senders. So why does Spamhaus continue to believe that their resources should be spent blocking legitimate commercial email where there is clearly a larger need to maintain focus on the criminal actors, as well as the diminishing needs by their ‘users’ to block legitimate (ie; dedicated and transparent) commercial emailers?
Because that “legitimate” spam you’re talking about is still painful for end users. If Spamhaus was doing consumers a dis-service, then receivers would stop using the Spamhaus list. Yet, they continue using it… I think that provides all the clarification I need that Spamhaus is doing good work.
15. [Not a question]
16. Can you confirm that spamtraps do not open, click or otherwise show engagement? In other words, if a client does have a spamtrap within their list, would removing or double opting in inactive subscribers help eliminate the trouble address?
It would be very bad form for a spam trap to process URLs in a message and open them. As you can imagine, someone trying to figure out which addresses are traps could simply send a whole lot of email with a bunch of unique URLs in each message, and then wait for the URLs to be queried by the trap collector. The URL hits could be correlated to determine the identity of the traps…
Visiting URLs in any sort of automated manner from trap traffic is a bad idea.
17. Does Spamhaus report traps hit immediately? For example, if a long standing client is reported for hitting traps, is it safe to say it was from a recent upload or signups?
Not necessarily immediately.
18. Besides typo, harvested, purchased, and recycled spamtraps, is there any other way a trap would appear in a client’s list?
None that I can think of.
19. What if someone manages to identify a spam trap’s identity and enroll it on a competitor’s mailing list? How lenient is Spamhaus to these issues knowing they exist?
If the competitor is using double opt-in, then it’s impossible for the spam trap to become enrolled in the competitor’s mailing list. To my knowledge, Spamhaus doesn’t click on the opt-in links for their traps…
20. Currently, we understand that typo-traps are being monitored by Spamhaus, but that they are mainly being used to advise marketers on the risks of mailing non-confirmed opt-in. Are there any plans over the next year to increase the blocking frequency and severity on marketers mailing to typo-trap addresses and domains?
I speculate that Spamhaus will increase the pressure on marketers to deal with typos somehow, so long as marketers continue to not get the message on this topic.
21. How many different types of spam-traps does Spamhaus monitor, and are some traps more dangerous than others?
Spam traps break down into essentially two types:
- Dedicated trap domains – these can be old expired domains that are picked up and registered anonymously, then allowed to “settle down” for a long period of time to ensure that no legitimate email would reasonably be sent to the domain’s users; or, they can be newly registered domains, from which trap addresses are created and then disseminated to spam lists via a variety of means (typically placing trap addresses on web sites to be “discovered”); and,
- Embedded traps – these are email addresses that are hosted on popular receiver services, which makes them hard to spot based on domain name alone.
It hardly matters how the trap address is created; what matters is whether your list management practices are so irresponsible as to result in trap addresses making it on to your list. Double opt-in, combined with regular communication with the list to verify validity virtually eliminates the possibility of getting a trap onto your list.
22. If a marketer is mailing to a purchased list of all actively engaged recipients (opening and clicking their emails regularly), do they still run the risk of hitting spam traps?
Yes, to the extent that the purchased list may contain spam trap addresses. I suppose that if the list seller could somehow prove that all of the addresses on the list recently showed activity, then the risk of hitting a trap would be reduced. But definitely not eliminated altogether. Purchasing a list is still not “best practices” because list recipients probably didn’t intend to receive mail from the buyer of the list when they signed up… This is going to lead to complaints.
23. Can you confirm that Spamhaus has a lower tolerance for newly allocated domains and IPs?
I would say definitely on this one. The age of domains and the reputability of the registrar are both very important indicators of risk to an email receiver. In the IP address world, receivers look at the sending history of the IP, its subnet, and the network (autonomous system number). The “newness” of an IP address is hard to establish; however, it’s not hard to establish that an IP has only recently started sending email. Traffic coming from a newly sending IP is definitely treated with suspicion.
24. Based on a sender’s business model, reaching out to their customers every 2, 3, or even 4 years may be necessary or applicable business practice. (example: purchasing a new car, TV, kitchen appliance). If this is necessary business practice, how can a sender do this safely without risking hitting too many traps?
I believe best practices is to reach out more frequently than once a year, requesting the recipient opt-in to the list again to continue receiving updates. I would suggest a quarterly reach-out, providing some valuable new information, and requesting a click to opt-in to further communication. For example, a company sending out warranty notifications could use the warranty mailing list to inform customers a) that they still have a warranty, b) that it is still valid, and c) of any updates to warranty servicing policies that are highly relevant to the customer.
If you used double opt-in to add the customer in the first place, then there shouldn’t be any problem hitting traps, so long as you prune the list if the recipient doesn’t continue to opt-in year after year.
25. What qualifies a domain for listing on the DBL? How is this different from listing the sending IPs instead on the SBL or CSS lists.
Spamhaus won’t reveal the precise list of things that qualify a domain for listing on the DBL. Generally speaking, if the domain is associated with spamming activity, then it may become listed. “Associated” could mean a number of things, including
- Being registered at a domain registrar that is known to register domains used for spamming, and who doesn’t respond to take-down requests;
- Being included in spam emails, or emails providing links to malware;
- Being associated with IP addresses that are used for sending spam.
26. What business hours do Spamhaus employees work? Or, what is the best time to reach out to Spamhaus?
Spamhaus is a global operation, with researchers across every conceivable time zone. I don’t think there is a best time to reach out.
27. Will Spamhaus ever engage in a phone-call with Marketers? [When asked for clarification, he said he means one-on-one calls with marketers who have gotten in trouble, or, say, a monthly conference call. I think the short answer is no for practicality and safety reasons, but maybe you can elaborate.]
This is doubtful – what information would Spamhaus usefully receive in a phone call that the marketer can’t communicate via email?
28. What information must be collected in order to provide evidence that a subscriber opted in to receive a commercial email?
I would question the usefulness of providing this information to Spamhaus. If your IP or domain have become listed, it’s probably because of a spam trap hit, and in that case, Spamhaus is unlikely to care that one subscriber was added via double opt-in, if clearly some other subscribers were added in another way. But, if you’re going to send anything to prove you are following best practices, then definitely the dates, times, and IP addresses involved in the double opt-in process would be a good starting point.
29. If an ESP sends mail for multiple clients on a shared range of IP addresses and uses a shared sending domain, what is the best way to work with Spamhaus to resolve a block listing issue for an offending client while maintaining service for the rest of the clients on the range?
If at all possible, send mail through a variety of different IP addresses and different reverse domains, and then separate your traffic based on your own intensive tracking of sender behaviour. You know more about your senders than Spamhaus does. Put the new senders on one IP address; high volume guys on another, etc. At the very least, this will hopefully keep the bad guys isolated so that the listing doesn’t negatively affect your good customers.
But, overall, if you want to have a successful ESP business, you need to get rid of the bad guys quickly.
30. If an ESP sends mail for multiple clients on a shared range of IP addresses and the sending domain for each is a separate sub-domain, what is the best way to work with Spamhaus to resolve an issue for an offending client while maintaining service for the rest of them?
The same advice as above. Sub-domains are not that useful. IP reputation is paramount, because IPs are in short supply, and impossible to spoof.
31. Is there any risk to having multiple, separate sub-domains of a single parent domain, each sending mail for different clients or are the domains treated entirely separately? (ex: branda.maindomain.com, brandb.maindomain.com, brandc.maindomain.com)
There is no easy answer to this, but I will suggest that registering separate top level domains costs more, and is therefore probably “better”.
32. Do they open/render images on emails they receive? If so, how would they expect a marketer to distinguish that from ‘real’ engagement?
I speculate that Spamhaus does not fetch image links, because that would permit senders to track opens by the traps and may lead to trap discovery. A small sample of such image URLs may be fetched, but certainly not every single one.
33. Ditto for clicks. Do they follow any of the links in the emails they receive?
See my answer above.
34. Are blacklistings all done by humans or are some automatically triggered by the receipt of *any* emails to an address? In other words, does the *content* or *purpose* of the message matter at all, or is it simply the fact an email was received? And if it is reviewed, are there formalized criteria for this evaluation?
The content or purpose of email messages sent to a trap is not important. The fact that you tried to deliver something to a trap exposes that your list management is broken. Listings on the CBL (and therefore the XBL) are driven entirely automatically, based on trap networks. The SBL is more manually driven; however, the input to the manual process is to a large extent trap activity.
35. Do they collaborate with other blacklist providers? E.g. is it possible to get listed (or a listing escalated) within Spamhaus because of ‘hits’ elsewhere or visa-versa?
I would suggest that cooperation between blacklists is minimal, for a variety of reasons.
36. Are decisions to blacklist made by any of the ‘volunteers’? is there a QC or review process internally?
I don’t know for sure, but I would speculate that all Spamhaus researchers — whether they are volunteers or paid — are able to make listing decisions.
37. Given that Spamhaus participants are all volunteers, how do they enforce consistent review and blacklisting behavior?
I don’t think it’s true that all Spamhaus researchers are volunteers. And in any case, I don’t see how this would really matter. The Catholic Church is volunteer driven, and yet is quite effective at being one of the largest and wealthiest organizations on the planet.
38. Why do they sometimes just list the offending IPs, but other times appear to name and attack specific marketing brands?
Because sometimes it’s effective to name the responsible brand rather than just the IP. Think of it this way: If only the IP is listed, then the brand can simply switch to a new ESP and get away with a few more blasts. If the brand is named, then the ESPs know they can’t take the brand on as a customer, or else they risk a listing. It’s a tool to enforce good behaviour in an efficient and rapid manner.
39. What do they say to claims they are unfairly targeting legitimate marketers?
First, define the term “legitimate marketer”. Is a “legitimate marketer” one that always uses double opt-in, never buys lists, and always sends email messages that recipients clearly want to receive? If this is the definition of a legitimate marketer, then the risk of a listing for that legitimate marketer is close to zero. The fact that you are using the word “targeting” indicates that you are probably not a legitimate marketer.
40. What’s their opinion of list rental or other one-time *opt-in* offers to an email address?
A list rental is not much different from a list buy; the recipients opted (if they opted at all) to receive one type of communication, and then ended up getting another. That’s spam.
41. Typos & errors happen. What thresholds is Spamhaus using to avoid accidental listings and/or what can marketers do to avoid?
I would imagine the thresholds are quite lenient in most cases, because Spamhaus has an extremely satisfying false positive ratio. If they didn’t, then receivers would stop using Spamhaus, and the project would fail.
42. Could they imagine cooperating with the DMA and if so, what would that look like?
I think you should ask the second question first. What value would there be to Spamhaus and email receivers to cooperate with an organization that actively promotes breaking best practices in order to get email delivered to unsuspecting consumers? Spamhaus is a sponsor and active participant in M3AAWG, and therefore I would recommend becoming a member of that organization and others like it if you wish to have face time with Spamhaus.
43. What can hosting networks do to get off Spamhaus?
Hosting providers need to track the sending behaviour of their customers using inline spam filtering technology. They should also actively monitor feedback loops, and apply best practices when vetting new customers. Customers who look bad to begin with, or turn bad based on metrics, need to be throttled back or kicked off the network. The positive result of this for the hosting provider is that spammers will tend to avoid even trying to sign up for an account on the network. Eventually, this leads to a reduction in credit card chargebacks, and of course better delivery rates for the good customers. Everyone wins.
44. I run abuse for a hosting provider in the US. We’ve had our share of SBL and XBL listings, and have responded by tuning in to feedback loops and aggressively removing customers who trigger listings and complaints. We also thoroughly vet new customers using a credit card fraud service as well as telephone verification, captchas, and other techniques. With all this being said, the problem is that mail still flows out of our customers’ servers (which we don’t control, because they are dedicated and VPS servers). How can we block the spam proactively? Is there a way that Spamhaus could send us feedback data other than a blacklisting? Can anyone else help with this?
Same answer as the above, with a focus on inline transparent SMTP filtering.
45. How has your business, mission, and the industry of blacklists changed over time? We first started working with Spamhaus in year 2000 and found that Spamhaus only listed networks that were known for sending majority spam, with very little legitimate email being blocked. As the years have gone by, it seems that Spamhaus is taking a more aggressive approach by listing some networks that send all opt-in email and their only flaws are typos and being single opt-in. Is our perception off? Where does Spamhaus see the future and how might that change over time?
Spamhaus has had to adapt over time to the changing practices of senders. Networks need to be listed occasionally in order to encourage the right behaviour – such as kicking out a bad customer who is snow-shoeing. A listing that seems only peripherally related, such as the listing of Cloudflare for permitting malware hosting a bit too liberally, is important because it provide a financial incentive to the organization to deal with the problematic behaviour.
46. Spamhaus has always been clear on recommending Confirmed-Opt-In email address collection. I am sure you know most legitimate mailers, including large corporations use single opt in. Is it part of Spamhaus’s mission or intention to blacklist list all companies that do not use confirmed-opt-in? How does Spamhaus determine which companies to list and which ones to not list? Many fortune 500 companies do not use confirmed-opt-in and most are not listed by Spamhaus. Does Spamhaus fear they could lose credibility by listing companies like GAP and other who play by most of the right rules with only typos and single opt-in being the only tarnish on their record?
Spamhaus will continue pushing for confirmed opt-in (i.e. double opt-in) until the end of time. It’s the only way to reliably guarantee that the recipient really does intend to receive communication from the sender.
47. Most consumers are not used to getting confirmation messages when they sign up for an email list. Unless consumers receive the confirmation right away, they are afraid to click on emails they don’t recognize for fear or phishing, viruses, and so forth. Even those that do receive the confirmation right away could be weary. I believe this is one of the reasons that legitimate companies do not use confirmation messages. How does Spamhaus suggest companies handle this? Before it becomes commonplace, there needs to be a tipping point to get consumers used to seeing and acting on confirmation messages. When does Spamhaus see this tipping point happening? In the past 13 years, I have not seen the majority of the marketplace adopt confirm opt-in.
This is a lame excuse for not using confirmed opt-in.
48. Lastly, we ask that Spamhaus be more clear when describing each section and also when responding to some of their listings. Spamhaus SBL in our experience is very responsive and easy to work with. Our concern is with the CBL (Composite Blocking List). The CBL web page says they only list IPs with spambot or virus like activity. It does not clearly explain that the CBL also operates spamtrap that can list legitimate mail servers IP. We once were listed for two weeks while we researched what could have been causing the issue (looking for misconfigurations, virus like activity, etc.) only to learn that the CBL administrators were upset and listed some of our IPs because they received one of our emails to their spamtrap. CBL administrators were not clear about this when we reached out to them as to what the problem was. They replied with terse replies like “This needs to stop”, but not explaining what needs to stop (was it a header problem, a spam problem, etc.). Please have the CBL administrators be more clear on if listings are caused by virus/bot like activity or if they were spammed. I am sure you know that a spamhaus listing is devastating to a marketer and yes, 60% of email bounces when blocked by spamhaus.
You need to understand that Spamhaus and the CBL are not large organizations with endless resources to deal with de-listing requests. They need to work efficiently, and focus most of their efforts on the core work of identifying badness.
49. What is the risk of a single “typo” email record? If the record is mailed once, but not ever again, is that enough to get listed? Is it true that a sender will get a warning first, and then if non active records are mailed again, that is when the block is placed? (If a person submits their email address, how can a marketer know if it’s good if we don’t mail it at least once?)
A single typo is pretty low risk today, but I can see that risk level rising over time as more stores allow customers to input their email address to receive a receipt. The best way to deal with this problem is to send an opt-in confirmation, or to rely on a separate authentication system such as allowing the customer to log in using their Google or Facebook account rather than manually entering in their email address.
50. Do Spamhaus volunteers take “complaints” from other people, or are they only identifying “bad actors” based on personal receipt of a message?
This is unknown.
51. How many volunteer complaints are required to flag a sender? (One? Ten?) Is this tracked at the individual level or just total? For example, one volunteer who complains five times counts as one or five?
I suspect Spamhaus is not going to share the answer to this. But I also point out that you don’t understand how Spamhaus volunteers work. They’re not so much volunteers, as hard working security researchers, who are highly trusted and skilled. They are “paid” in the satisfaction of dealing with a very large problem and making a huge difference for hundreds of millions of people every day.
