Home / Blogs

Challenges in Anti-Spam Efforts

Without commenting on the particulars as they relate to Goodmail—especially since I am on the advisory board for Habeas, a competitor—let me note that public discussion is largely missing the nature of the current Internet mail realities and the nature of the ways we can deal with them.

There are two articles in the current issue of the Internet Protocol Journal, of which I wrote one, that provide some useful background about this reality.

Simply put, Internet mail needs to sustain spontaneous communications—that is, communications without prior arrangement—and the benefit of such a capability is fundamental. However the scale and diversity of the modern Internet now includes many folk who the security geeks appropriately call Bad Actors. We are stuck with these competing points: Maintaining open contact, but dealing with some very nasty users.

A great deal of very good work has been done, to detect these bad actors and their bad messages. Often, that work is quite helpful. In spite of this the total amount of global spam and email abuse has yet not gone down. We must continue with efforts to detect and deal with Bad Actors, but there is a separate path that is at least as valuable:

We need methods for distinguishing Good Actors. Folks who are deemed “safe”. In effect, we need a Trust Overlay for Internet mail, to permit differential handling of mail from these good actors.

In general terms, a trust overlay requires reliable and accurate identification of the actor and a means of assessing their goodness. In other words, authentication and reputation.

We are already pursuing a standard for message transit handling authentication, through Domain Keys Identified Mail (DKIM). There is discussion about various assessment standards for reputation and accreditation. Although DKIM is quite viable in its pre-standards form, there is no candidate for standardized reputation reporting.

With all of this as background, imagine that you are an online service that needs to ensure that a customer order confirmation, or an equivalent critical transaction message, is delivered to the customer. Then imagine that you are offered a means of safely and reliably identifying this specific class of mail, so that it receives differential handling. The incentives for a company to pay to ensure that delivery are substantial.

And that is what the recent announcement is about. It concerns a means of ensuring delivery of “transactional” mail. This is quite different from “marketing” mail and it is not in the least controversial.

I would greatly wish that the mechanisms used open standards, but the basic model of developing a trust-based overlay for Internet mail seems an essential enhancement.

By Dave Crocker, Consultant

Filed Under

Comments

Justin Mason  –  Feb 9, 2006 10:26 PM

Hm.  If you read the press, such as the ClickZ article, the NY Times, USA Today,  and eWeek, there’s absolutely no indication that this is intended only for transactional mail; in fact, the USA Today article discusses it in terms of ‘advertisers’ paying for stamps.


And that is what the recent announcement is about. It concerns a means of ensuring delivery of “transactional” mail. This is quite different from “marketing” mail and it is not in the least controversial.

‘Transactional’ mail may be one part of what they want to do, but it’s certainly not all the announcement was about—at least, not going by what they’re saying on the record.

Dave Crocker  –  Feb 10, 2006 3:05 AM

Jason, et al,

Indeed the media reports have been impressively confusing.  And I gather that there is no archival, official press release to consult. So I am going by the follow-up postings from the AOL and Yahoo folks, on various anti-spam lists. 

Independent of the AOL/Yahoo intent, I was trying to underscore the distinctive nature of transactional mail, as well as just how uncontroversial it should be to use a third-party vetting service by the originators of such mail. 

/d

Eric Thomas  –  Feb 10, 2006 6:45 AM

The original AOL announcement that started all this can be found here:

http://www.the-dma.org/cgi/dispnewsstand?article=4405

As you can see, it was not about transactional e-mail but, in practice, about bulk e-mail. Being on the AOL whitelist is a must if you send a lot of mail.

AOL eventually backed off - sort of, they claimed they never made that announcement in the first place. They aren’t saying that the journalists lied, the party line seems to be that the announcement was an internal memo that should never have been sent to the press. That did not exactly help anyone sleep tighter at night. You can read a summary a:

http://www.webpronews.com/topnews/...

I think you’ll find that most of the people who participated in the outcry don’t have a problem with the concept of third-party vetting services per se. As one of the most vocal people, I will tell you up front that I think vetting is a great idea,

provided

that it is implemented using open standards to promote competition. And it doesn’t need to be very complicated - a simple DKIM extension to locate the public key at the vetting company, rather than at the sender, would give you most of what is necessary, plus rapid implementation and deployment.

I would be happy to pay a reasonable fee for vetting. $10,000-40,000 per day is not a reasonable fee for my modest operation. I have a broad blend of customers, including many non-profits that cannot dream of raising that kind of money, let alone spending it on e-mail delivery. The vetting costs obviously do not come close to this figure, and this is why I want to see open standards and competition. In this context, I would applaud vetting. I do not want to sign a blank check, but within reason, I would see to it that LISTSERV became an early adopter.

For the record, I think a reasonable fee level is what we pay for SSL certificates, and I think this can be achieved with technology that either exists or can be developed rapidly.

  Eric

Suresh Ramasubramanian  –  Feb 10, 2006 3:40 PM

Eric, there’s a minor problem here. A lack of understanding of terminology that AOL people used. Quoted well out of context, In a lobbying group where the people participating in it dont like spam filtering all that much, and see it as a hindrance in (as their former chief bob weintzen put it) emailing a coupon for Tide detergent to every household in America.

And then have an article based on wrong assumptions and misinterpreted facts pushed all around the media.  With mass hysteria about a big corporation stiffing people who want to mail their users.

The EFF hasn’t helped at all with Cindy Cohn rambling on about how this is a way to “shake down” non commercial email list operators like Dave Farber, or Aunt Tilly sending out her christmas greetings, into buying a online stamp for whitelisting. I’ve got a detailed set of thoughts on this for circleid that should be posted shortly.

This post (on .(JavaScript must be enabled to view this email address)) by Christine M Borgia, AOL’s technical manager for antispam operations, should help explain some of the context.

srs


————Original Message————
Subject: MISC: AOL, the Enhanced Whitelist, & CertifiedEmail
Date: Thu, 9 Feb 2006 18:26:01 -0500
From: Christine M. Borgia

To: .(JavaScript must be enabled to view this email address)

There have been several requests here for clarification from AOL. I hope I can address a lot of the concerns and confusion of the last week around the comments linking the Enhanced Whitelist to the introduction of CertifiedEmail.

1) At least one article made it sound like the AOL whitelist is being
changed. This is not true.  The AOL whitelist will continue as it exists today, free of charge.

2) We are making changes the EWL. Over time the EWL has grown beyond its original intent, which was to reward mailers who could meet stringent requirements over a period of time, with enhancements beyond the regular whitelist. As the EWL grew in it’s scope, we began to see it abused. This happened often enough that it became an operational concern to us. Starting last fall we began a process of incremental changes to get the EWL back to what it should be.  This process will continue over the next few months. We anticipate being done with the tightening of the requirements by early summer. There is obviously some confusion that this schedule implied that the EWL was being discontinued.  This is not the intention.  The EWL is like any other operational tool that AOL and others utilize - we are constantly
looking to tune functionality, make them more effective, and ensure that they are serving the purpose for which they were originally intended.

3) CertifiedEmail is a fact at AOL and we believe it will be a success.
Contrary to the reports I’ve seen, we are not forcing anyone to pay for the inbox. We believe CertifiedEmail provides great benefits to the mailer and we believe it will benefit AOL members, giving them a level of trust in commercial mail they simply can’t rely on now. There are many benefits of CertifiedEmail to commercial mailers above and beyond the current limitations of the EWL.

The goal of these changes is to provide the best member experience for
AOL users. We will always evaluate and tweak our processes to achieve
this goal.

Thanks,
Christine

Dave Crocker  –  Feb 10, 2006 10:34 PM

Folks, the major problem here is continuing to focus on what AOL did or did not say.

Instead, the discussion should be about: the need for a trust overlay on email, what it should look like and how it should function.

Really, let’s put our energy into something that will move us closer to a goal of distinguishing mail that is known to be acceptable from mail that might or might not be acceptable.

/d

ps. Sorry for all the bolding, but emphasis is exactly the current problem.

Justin Mason  –  Feb 10, 2006 11:11 PM

‘let’s put our energy into something that will move us closer to a goal of distinguishing mail that is known to be acceptable from mail that might or might not be acceptable.’

Agreed.  And I also agree that reputation services built on open standards, like DKIM, are a much better way to get there.

Eric Thomas  –  Feb 11, 2006 1:43 AM

Dave,

The reason I made a concrete technical suggestion at such an early stage (extending DKIM to allow the public key to be at the vetting company rather than the sending company) is that I think we need to get moving as soon as possible. Building something on top of DKIM will save that much time.

For while I agree that what happened at AOL is of secondary importance, the existence of this AOL memo suggests that, at a minimum, there have been discussions about a rapid transition to a single-provider vetting service. Even if that was not the case, a lot of people have now been exposed to that idea as a result of the press articles, and it is likely being discussed in boardrooms. Some journalists see e-mail stamps as a done deal and are asking what service I think will be charged for next. I would like to tell them that we have an open standard alternative coming up, but I cannot honestly make this claim. The sooner we have something to show, the better.

  Eric

Suresh Ramasubramanian  –  Feb 11, 2006 4:07 AM

My article in circleid on this issue -
http://www.circleid.com/posts/...

Dave Crocker  –  Feb 11, 2006 7:36 PM

Eric, et al,

The reason I made a concrete technical suggestion at such an early stage (extending DKIM to allow the public key to be at the vetting company rather than the sending company) is that I think we need to get moving as soon as possible. Building something on top of DKIM will save that much time.

Well!  Now we can get into something of real substance.  Thank you!

Although DKIM represents relatively mature, basic technology (version 3 of domainkeys), it is relatively new to the larger public forum.  Consequently, most energy has been spent on its basic characteristics and details.  Indeed, that is where the focus should remain, in order to make the quickest possible progress in producing an official standard. 

(However, please forgive me for making a plug about the pre-IETF version that is already deployed for production use.  Take a look at DKIM’s site. Existing implementations are using draft-allman-core-01 as a common specification base.  From my experience, I will be quite surprised to see a standardized IETF version appear in as little as a year and not at all surprised if it requires as much as 3. In order to get benefit sooner than that, I am recommending the community use the pre-standards version.  It is the result of an extensive, multi-vendor effort and it is already in field use.)

Now to your own point:

There are different scenarios possible, for using DKIM.  It has some subtle characteristics that provide more flexibility than is obvious.  So in fact, the signature can be with a key from the vetting agency!

Here’s how it could work:

The vetting agency assigns an originator a specific DKIM “selector”.  A selector serves as a qualifier for the domain name being used for the signature—note that that name need not match the RFC2822.From or RFC2822.Sender fields. The qualifier is added to the domain name, during the DNS query, so that signing attributes can be specific to that selector.

So, a vetting service with a domain name of “vetting-example.com” could give a selector, such as example-orig, specific to a customer Example Originator. The originator would sign under the domain name and selector and recipients would query the DNS for signing information under the domain name:  example-orig.vetting-example.com.

A vetting agency can give different selectors to different customers and can retain real-time control over the DNS query results.

I believe this is a style of use that is a bit similar to the Goodmail model, but note that it is entirely optional and open, rather than proprietary, both in terms of technology and in terms of permitting multiple vetting services.

So, Eric, I believe that your thinking is not only valid, but that DKIM already can support that style of use!

/d

Miles  –  Feb 12, 2006 9:10 PM

Even more options:
- the vetting agencies selector could be part of the RFC2822.From or RFC2822.Sender’s domain path, but name served by the vetting agency:  vettingagency._domainkeys.domain.com, which would allow them to serve key.vettingagency._domainkeys.domain.com
- the vetting agency could publish a list of domain/selector pairs in a DNS zone just like they do for IP addresses today—for instance transactional.vettingagency.com

Comment Title:

  Notify me of follow-up comments

We encourage you to post comments and engage in discussions that advance this post through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can report it using the link at the end of each comment. Views expressed in the comments do not represent those of CircleID. For more information on our comment policy, see Codes of Conduct.

CircleID Newsletter The Weekly Wrap

More and more professionals are choosing to publish critical posts on CircleID from all corners of the Internet industry. If you find it hard to keep up daily, consider subscribing to our weekly digest. We will provide you a convenient summary report once a week sent directly to your inbox. It's a quick and easy read.

Related

Topics

New TLDs

Sponsored byRadix

Threat Intelligence

Sponsored byWhoisXML API

Cybersecurity

Sponsored byVerisign

Domain Names

Sponsored byVerisign

DNS

Sponsored byDNIB.com

IPv4 Markets

Sponsored byIPv4.Global

Brand Protection

Sponsored byCSC