There has been a lot of recent discussions and questions about reputation, content and delivery of email. I started to answer some of them, and then realized there weren't any basic reference documents I could refer to when explaining the interaction. So I decided to write some. This post is about IP address reputation with some background on why IPs are so important and why ISPs focus so heavily on the sending IP.
In a presentation EU Commissioner Viviane Reding gave a preview of the new Privacy regulation her DG is preparing. As she states, privacy rules need to be brought up to date and harmonized. With all 27 member states having the same rules and tools to enforce, a company only will deal with one privacy commissioner... So, what if we, for the sake of this blog, take this initiative towards spam and cyber crime. What would this do to spam enforcement?
While Canada was dragging the chain when it came to introducing anti-spam legislation, it is now making up for lost time. Ottawa's new law -- expected to be operational early this year -- has severe fines for violations and is viewed by some as too tough. Known as CASL, the new law aims to crack down on spammers and mailing list companies but in doing so, tightly regulates the way businesses can market to prospective customers via email and online.
IBM published a video where it predicts what the world will look like in 2016 (see bottom of this post for the link). It includes the following five predictions. I want to start with the last one -- that junk mail (i.e., spam) will disappear. You'll need to watch the video to get the nuances of the prediction, but IBM says that in five years, Junk Mail will become a thing of the past.
I was reading in the Canadian Lawyer Mag that businesses in Canada are now coming to grips with the Canadian Antispam law that was passed last year. Canada's antispam law is much tougher than most jurisdictions. Aside from the penalties of the law, which are steep, what differentiates it the most from the US law is that Canada's law is an opt-in law; marketers who send commercial email must be able to demonstrate that they received consent in writing in order to market to people. As expected, people who are most affected by the law - marketers - are upset about the lack of wriggle room and how it could affect their business.
For email usage, abuse reporting requires cooperation between senders and receivers. That's why RFC 5965 specified a standard format for it. However, Wikipedia lists only 18 feedback providers today. It is often said that the number of legitimate mailbox providers in the world is rather small, possibly some hundreds of thousands, but certainly more than that.
An interesting new paper from the Naval Postgraduate School describes what appears to be an interesting new twist on spam filtering, looking at the characteristics of the TCP session through which the mail is delivered. They observe that bots typically live on cable or DSL connections with slow congested upstreams. ... This paper tries to see whether it would be practical to use that info to manage spam in real time.
In my last post I blogged about greylisting, a well-known anti-spam technique for rejecting spam sent by botnets. When a mail server receives a an attempt to deliver mail from an IP address that's never sent mail before, it rejects the message with a "soft fail" error which tells the sender to try again later. Real mail senders always retry, badly written spamware often doesn't. I found that even though everyone knows about greylisting, about 2/3 of IPs don't successfully retry.
Greylisting is a hoary technique for rejecting spam sent by botnets and other poorly written spamware. When a mail server receives an attempt to deliver mail from a hitherto unseen sending host IP address, it rejects the message with a "soft fail" error which tells the sender to try again later. Real mail software does try again, at which point you note that the host knows how to retry and you don't greylist mail from that IP again.
Mainsleaze is nerdy slang for spam sent by large, well-known, otherwise reputable organizations. Although the volume of mainsleaze is dwarfed by the volume of spam for fake drugs, account phishes, and Nigerian 419 fraud, it causes work for mail managers far out of proportion to its volume... The problem with mainsleaze is that it is generally mixed in with mail that the recipients asked for, and there's no way to tell the difference mechanically.