CircleID recently interviewed Meng Weng Wong, the lead developer of Sender Policy Framework (SPF) and founder of Pobox.com. As one of the leading anti spam authentication schemes, SPF is used by companies such as AOL, Earthlink, SAP and supported by anti spam companies such as Sophos, Symantec, Brightmail, IronPort, Ciphertrust, MailArmory, MailFrontier, Roaring Penguin Software, and Communigate Pro. Last month, Microsoft announced its agreement to merge Caller ID, its own proposed anti spam authentication scheme, with SPF—the joint standard is called ‘Sender ID’.

In this two-part interview, Meng Wong explains how SPF got started, where it is today and what could be expected in the future of email.

CircleID: A lot of attention has been paid to Microsoft’s recent announcement about an agreement to combine Caller ID, its anti-spam authentication method, with Sender Policy Framework (SPF), another popular anti-spam authentication scheme. Can you share with us some background about your involvement with SPF?

Meng Wong: In 2002 Paul Vixie, the brains behind BIND, wrote a short paper titled “Repudiating Mail-From”. That inspired two other proposals, “Reverse MX” by Hadmut Danisch and “Designated Mailer Protocol” by Gordon Fecyk. In late 2003 I combined the best of both proposals and called the result “SPF”. (I picked the acronym first and decided what it stood for second, which is why old references call it Sender Permitted From. Now it means Sender Policy Framework, which is a touch more accurate. Yes, references to sunblock are intentional.)

I presented the SPF concept at YAPC and (O’Reilly Open Source Convention) OSCON in 2003. Audiences liked the idea, and the mailing list kept growing. In November 2003 at the Hackers Conference I met Mark Lentczner, my co-author and an absolute genius at protocol design. During the conference and in the following weeks we rewrote the spec together. Eric Raymond talked about SPF at the MIT Spam Conference in January and I presented it before the IETF in Seoul in March.

My role in the SPF community is Chief Communicator: I run the website and go to conferences to spread the word. I also wrote Mail::SPF::Query, the reference library.

CircleID: Who is using SPF today and what level of success has it achieved in the fight against spam?

Meng Wong: In the six months since we declared a spec freeze, 20,000 domains have self-reported at the online registry; that’s probably a fraction of the true total. Lots of domains have published records, including AOL, Amazon, Google, O’Reilly, SAP, TicketMaster, Mail.com, w3.org, Earthlink and Verizon. And the ones who haven’t published are working on it.

We expect adoption to pick up exponentially; according to some estimates, the number of sites checking SPF doubles every three weeks. SPF plugins are available for all the major Mail Transfer Agents (MTAs).

CircleID: SPF, as we understand it, is aimed at addressing a fundamental weakness in Simple Mail Transfer Protocol (SMTP). Could you give us some insight on how this anti-spam authentication works? How does it differ from that of Microsoft’s Caller ID?

Meng Wong: Suppose you get a letter in the mail. The return address on the envelope says “Amazon.com”. You know that Amazon is based in Seattle, so if the postmark says “Seattle, WA” you could believe it’s really from Amazon. But what if the postmark says “Nigeria”? That’s suspicious.

The key insight in Vixie’s original paper was this: domains publish MX records to tell senders where to send incoming mail. Domains can also publish records to tell receivers where to expect outgoing mail to come from. This kind of thing is generally called a Designated Sender Scheme, because domains designate their outbound mail servers. RMX, DMP, SPF, and CallerID are all Designated Sender Schemes.

Here’s how it works:

Suppose example.com receives mail at 192.0.2.1 and 192.0.2.2. When it sends mail, it uses those two servers, as well as 192.0.2.3. Example.com would publish an SPF record that said: “v=spf1 ip4:192.0.2.1 ip4:192.0.2.2 ip4:192.0.2.3 -all”.

When a mail server gets mail that claims to be from someone at example.com, that server can fetch example.com’s SPF record and see if the connecting SMTP client is designated.

There are a number of shorthands available: for example, “v=spf1 mx” means “I send mail from my MX servers”.

Vixie’s paper talked about running the authentication check against the domain of the MAIL FROM parameter. Its descendants RMX, DMP, and SPF all use the return-path identity.

Some of the proposals that were presented to the IETF use different identities: instead of the return-path, those proposals attempt to authenticate the HELO argument, or the PTR of the client IP address. The SPF community has recently developed a Unified SPF model that embraces all of the above semantics.

CallerID defines an algorithm to extract from the headers what it calls a Purported Responsible Address. The PRA starts out as the “From:” header, but if there’s a “Sender” header present, “Sender” will override. If “Resent-From” is present, that overrides Sender. The algorithm was developed empirically; it is meant to pull out the last logical hop from the headers. If a message was forwarded through a service like acm.org or pobox.com, the PRA should be the alias at the forwarding service that was responsible for causing the final delivery.

SenderID applies SPF checks against the PRA. SPF Classic applies SPF checks against the return-path. In both cases, though, the DNS record that’s used in the checks is generally in SPF format. That’s why I sometimes talk about “SPF records” and “SPF checks”, even when the authenticated identity is different from the return-path.

CircleID: What are the advantages of having the two methods merged? What are the key measures that are required in order to make this collaboration successful?

Meng Wong: In May, Microsoft agreed to adopt SPF’s syntax and semantics for their Caller ID proposal. The result, which we’re calling SenderID, applies SPF checks to the Purported Responsible Address from the message headers. I believe it represents the best of both worlds. The best thing about the CallerID proposal was the use of XML, which provides a path to future extensibility. SenderID uses a dual-syntax model: because the core semantics are drawn from SPF but are expressed in XML, SenderID clients will be able to parse both XML and regular SPF records. This means that if, in the future, the SPF syntax runs out of steam, and we need to say things that can’t be said in the SPF language, we can upgrade to XML with a minimum of pain.

Meanwhile, SPF Classic, which applies SPF checks to the return-path, is still going strong. It is more mature and is better understood; it continues to enjoy a lot of support in the open community. Many people on the MARID mailing list have argued that SPF Classic and SenderID can and should proceed in parallel, especially since they both use the exact same records but can be used for two mutually compatible purposes.

CircleID: You have been quoted in the media, saying:

“In the last six months or so, there’s been a fair amount of uncertainty (about sender authentication). These are two very similar proposals and do many of the same things. The big players have been telling us ‘When you get your story straight, we can go ahead,’ but until that happens, people have been waiting to see what happens”

What sorts of issues have caused the above mentioned uncertainties, particularly among the"big players”?

Meng Wong: The big players wanted CallerID and SPF to converge because they were both designated sender schemes. It didn’t make sense to have two different syntaxes and two different semantics for two different identities, because receivers would have to code up two sets of functions that do basically the same thing, and publishers would have to publish two records that say the same thing. Now that Microsoft has committed to remain backward-compatible with the SPF syntax, publishers don’t have to worry about publishing both, and receivers don’t have to worry about parsing both.

Read Part II of this Interview.