Mike Hammer’s thoughtful article, A Few Thoughts on the Future of Email Authentication, should trigger thoughtfulness in the rest of us. Email abuse has been around a long time. Anti-abuse efforts have too. Yet global abuse traffic has grown into the 90+% range, with no hint of trending downward. The best we hear about current effectiveness is for last-hop filtering, if you have the money, staff and skills to apply to the problem. No impact on the source, never mind the transit infrastructure. As constructive as Mike’s article is, I think it demonstrates a number of the core problems the anti-abuse community is having in being more effective. Since Mike has substantial experience in this space and thinks deeply about this topic, he’s not the target of what follows. The rest of you are…

The challenges are at every level of discussion: terminology, mechanical detail, individual behavior, social forces. In spite of the best of intentions and serious effort, we really aren’t very careful with our thinking or discussion. Abuse is a system-wide social problem but we treat it as a localized, technical phenomenon. Note for example that the only pattern of improvement has been in (local) last-hop filtering; besides continuing to waste massive amounts of bandwidth and computation, it ensures an arms-race. At the least we need to change the game and find ways to put pressure on the sources. When we purport to be taking the broader view, what we really tend to mean is “if all the Good Actors will merely (immediately) adopt my model for a perfect solution, then everything will be fine.”

We need to use language that has precise meanings and is shared widely among the anti-abuse community. We need to define system-wide mechanisms that have incremental benefit, starting with adoption by only a few. We need proposals that start with models of human and social behavior that reflect the messy, non-conforming, statistical real world. We need to stop talking about how wonderful things will be when all the Good Actors have adopted a particular mechanism. It won’t happen. It never does. Adoption of any Internet-wide mechanism is very slow—measured in years and even decades—and tends to be very messy, especially when trying to change an existing service. Anti-abuse has demonstrated well that it is no exception. Deal with it.

Terminology, mechanics and Messy Scenarios

Among the terms that we toss out with indiscriminate abandon, my favorites are: sender, originator and From. Do we mean the author, the author’s organization, one of the MTAs at the author organization’s ISP, a third-party under contract to the author’s organization, the machine that just handed me the message, or… someone else? Discussion almost never is careful about this. So, for example, when Mike says “At the transport layer we have now have SPF providing a linkage between the RFC5321 “Mail From” address and the originating IP address” which of the actors in the email process does “originating” mean? Perhaps you are expert enough to know that SPF uses the IP Address of the SMTP client that is the immediate neighbor (last hop) before the evaluating receiver. Perhaps you aren’t, but instead think it means the real originator. Hmmm. Real originator. Do I mean the author or their organization’s first MTA or…?

Most participants in anti-abuse discussion really do have a clear sense of what they mean when they use a specific term. The only problem is that there is no way for the listener to be sure what that is, because one speaker’s definitions are likely to be quite different from another listener’s. This lack of shared terminology and even lack of a shared, precise email system model is what prompted me to start work on Internet Mail Architecture to document the existing service, which is finally going through formal IETF standardization approval. Although documenting an existing service ought to be easy, it has taken five years to complete and required a surprising amount of post-hoc invention and email technical community collaboration. Email really has gotten quite complex, so our discussion of it needs to be far more precise and nuanced.

One reason we often think of email as simple is because our own world tends to involve only a few, specific email usage scenarios. Each scenario tends to be rather straightforward. Therefore, email is straightforward? Not.

Email is a general-purpose service, satisfying needs ranging from brief, single, pair-wise exchanges among strangers, to decades-long exchanges among spice (spouses?), to essential inter-organization group discussions, to contract negotiations, to mass-market bulk-mailing campaigns, to… General purpose human communication services must satisfy infinite vagaries in usage style. In our anti-abuse vigor, we often propose schemes that will close off one or another valid scenario. Yet such cavalier, Procrustean restrictions are likely to cause real social damage. The fact that you and your organization do not need a scenario is no excuse for depriving others of it. And if what you really mean is that your proposal seeks to define a separate, specialized service, then say so. There is nothing wrong with considering new, independent services that could have desirable properties, but that’s quite different from claiming to propose changes to an existing service. And admitting that it’s separate changes the basis for evaluating who will adopt it—and who won’t—and why.

And by the way, we need to stop making proposals premised on coercion or enforcement. We do not get to impose the Final Ultimate Solution to the Spam Problem. (Let’s skip over the fact that we won’t ever find a real FUSSP.) Internet adoption is by attraction, not coercion. So, “who will adopt?” needs to have an answer that specific, immediate, pragmatic, personal and compelling.

Can’t We All Just Get Along?

A service that exists for the sole purpose of human exchange has to be counted as cooperative, but you’d never know that about email, by watching different communities of anti-abuse workers interact. Anti-abuse discussions are dominated by major email marketing campaign senders and major ISP receivers. The senders tend to focus only on doing whatever will get their mail delivered which, after all, the addressees want (they say.) So ISPs are unreasonable to present barriers to bulk mail? Not. ISPs tend to focus on complaints from their customers. Mail from marketing campaigns cause complaints and, therefore, bulk mail senders are a problem? Not. We need to stop this fundamentally antagonistic relationship between sources and sinks. I’ve started to think that ISPs and bulk mail senders need some marriage counseling.

The real test of collaboration is handling disagreement. That includes disagreement about goals, as well as disagreement about the details. One way to handle disagreement is to find a core point about which both sides can and must agree. Here’s my suggestion: On both sides of the email exchange equation, folks need to cast things in terms of their real client, even when that client isn’t footing the bill. The real client is the recipient, exactly as Mike said, “This is not simply about protecting brands, companies or deliverability. It is about protecting end users, their personal computers, the organizations they work for and society as a whole.”

Recipients do want (some) mail. When trying to negotiate for delivery (and viewing) why not start with the recipient’s perspective and treat the sending and receiving operators merely as their agents? For any particular scenario, start with questions about the recipient. How do we know they want a particular piece of mail? How will they react if they don’t get it? If a clear case cannot be made for delivery (or rejection) then there is something wrong with the underlying approach.

Authentication is Easy, Trust is Hard

Early anti-abuse work focused on detecting Bad Actors and bad content, without the cooperation of the author/sender. It is a mature and effective area of endeavor, but it remains a losing long-term proposition because it is heuristic and because it ensures a continuing arms race. Hence we now have the additional focus on developing trust-related mechanisms, with some collaboration among sources and sinks. In that realm, we have spent a large fraction of a decade focusing solely on enabling mechanisms for validating identities. While authentication is essential, it is by far the simplest part of a trust infrastructure. When you have high-quality authentication in place, you have nothing useful. Yes, I said “nothing useful.” Really.

For authentication to be useful, you also need some sort of evaluation mechanism, whether an ad hoc, private whitelist or a trusted, third-party assessment service. Authentication is only one component of a trust service. This, of course, leads to a chicken-and-egg problem trying to get adoption by parties who might not see concrete benefit anytime soon. While the mechanics and operation of authentication are well understood, they aren’t cheap to implement. Absent an immediate value proposition, why should an organization go through the expense? Operations folk are not usually swayed by vague promises of eventual benefit. So what are the specific, immediate assessment, whitelist, reputation, certification benefits available for an adopter of DKIM or SPF? Absent a meaningful assessment mechanism, the answer is: none.

For starters, let’s stop treating authentication as a means of solving abuse. (Until we figure out how to make all humans act with good intention, we will never solve abuse of email or anything else.) Authentication is about giving benefits to members of a trust club; it is not about being nasty to the untrusted rabble outside the club. Most discussions about using authentication lean towards using it to find abuse. But it can’t do that. Or at least it can’t do that as a primary function. If there is enough adoption of authentication, then a derivative capability might include treating unauthenticated messages differently. But if that capability is a primary goal, we’ll be waiting another decade before we see any benefit. We need, instead, to take an authenticated message on its own terms. Here it is, sitting in front of you. It gives you an identifier you can believe. Ignore messages to the left or right. They are distractions. What can you do with just that message, just that identifier, and some sort of assessment information?

Answer that question in a way that is easily developed and deployed. Make sure the first kinds of assessment are cheap to provide and afford meaningful help with each use of it, even when there are few adopters, and you will finally be offering a value proposition that makes sense for early adoption.

Two, Near-Term Approaches

Independent assessment services like Spamhaus, Return-Path, and Goodmail (for whom I am an advisor) are adding domain-name authentication into their mix. Serious, third-party certification is real work. It’s loaded with technical, administrative and legal burdens. Nothing about it is simple or immediate. While third-party certification is essential for the long-term, we can’t pin our hopes for Internet-scale adoption of authentication on their success. Are there, perhaps, more modest approaches for using authentication?

Two come to mind. The simplest is application of classic Bayesian content analysis, to develop a reputation history for a particular identifier. Perform the usual types of statistical evaluation of a stream of messages having the same signature. You will quickly formulate an assessment. If your assessment is negative, you are in the unusual position of knowing who to complain to: Since the message stream is authenticated, there is an explicitly and reliably specified responsible party. If your assessment is positive, you can start treating that stream differently (and better) than messages lacking authentication.

The second approach involves a published list, but is much more modest than an all-out certification service. Affiliation Lists (AffiL) is a proposal that Jeff MacDonald, of e-Dialog, and I developed:

“Typically, an affiliation list merely documents an existing relationship, whereas quality assessment statements are the result of incremental and subjective work by the publisher. Hence, affiliations are more easily documented. In terms of using the publication information for performing decision-making by the consumer, the utility of an affiliations list is based on having the consumer of the list perform the quality assessment step, rather than the publisher.” (See specification)

Imagine a list published by the Federal Deposit Insurance Corporation (FDIC) of member organizations or one for licensed pharmacists. Membership already exists, even if it is not already published. Such a list would indicate domain names registered to a member. Anything signed under one of these domain names would mean that the signing organization had an affiliation that could be meaningful for a receiver’s message handling analysis. The lists do not certify email quality by the signer, but they do provide useful input. While membership would not automatically mean that a receiver should trust the message, it ought to mean a higher level of safety than stray messages coming in from the world of mistrust.

We’re still shopping for an organization interested in publishing such a list.