|
The diagram below depicts two mail transmitters relaying mail on behalf of two users each, and a target MX receiving that mail for four recipients. The difference between the two transmitters is how they deal with content filtering.
Why do we run content filters at the recipient’s side? Paul Graham’s Plan for Spam introduced them that way. After several years, we can say that plan doesn’t work very well. Email has become much less reliable. One way to recover reliability, at least between trusted parties, is to run filters at the sender’s side. Let’s look at the diagram in more detail.
Users are connected through authenticated and possibly encrypted connections, both senders and recipients. Some users are connected through the Internet, some directly to the relevant server.
The first MSA (?) relays according to current SMTP standards. The receiving server runs content filtering, but doesn’t know what to do in case an accepted message turns out to be spam. The amount of mumbo jumbo required for effective spam filtering is high, and may involve delays. If the message is considered spam, most times it will be silently dropped. This is where unreliability stems from.
The second MSA (!) relays according to the VHLO proposed SMTP extension. The sender knows how to handle spam, because it knows any required detail about the authenticated sender. The recipient trusts the sender, not because they have specific arrangements, but because the sender identifies itself, e.g. providing its domain registration reference, and relays for its own users only, at least for the illustrated session.
VHLO, Verified Hello, provides a reliable channel that can be used in parallel with existing EHLO traffic. It employs the authentication, authorization, and vouching techniques that have been developed during the past years, and allows postmasters to manage them, e.g. getting aware of what conditions a receiver MTA requires… But I’m not going to describe protocol’s details, that are being discussed. I want to ask: Are you ready to turn the table on spam?
Sponsored byVerisign
Sponsored byIPv4.Global
Sponsored byCSC
Sponsored byRadix
Sponsored byWhoisXML API
Sponsored byDNIB.com
Sponsored byVerisign
You will need some other property of that MTA (other than the fact that it issued VHLO rather than EHLO) to trigger the acceptance of mail from it. Otherwise, spammers will just use VHLO. Whatever property you need in addition to that VHLO verb could more easily be done with an EHLO extension. In particular, the AUTH extension seems to cover almost all of this case.
If that MTA is “trusted” and has a static ip address, then you could simply use a local whitelist on the receiver to get the same effect.
“The receiving server runs content filtering, but doesn’t know what to do in case an accepted message turns out to be spam”. In that case, the receiving server is broken. You should *almost never* actually accept (2xx response to DATA) a message and then make some spam/ham decision about it. Modern systems make that decision *before* returning the status code to the DATA command.
“The AUTH extension is for authentication by prior arrangement.”
My point, poorly expressed, was that it will be *far* easier to gain widespread acceptance of an ESMTP/EHLO extension than it will be to gain widespread acceptance of another SMTP verb. We already have ESMTP which allows the receiver to advertise extensions that it understands. Any data that you might want to pass with VHLO could be passed with something like “MAIL FROM:
VHLO=arguments”.
“and there are other points to be made in favour of accepting all mail (with post-classification of some kind)”
The work of classification still needs to be done at some point. If you do that work during the SMTP session, you will either accept and deliver the message, or the SMTP client will get an unambiguous rejection indication, so the sender will know that their message was not delivered, and they can arrange alternative means of communication. *Any* system that does that classification work after accepting the message *will* be unreliable unless the classification mechanism is perfect. When it makes a mistake, and classifies a wanted message as spam, then the sender will think it has been delivered, and the receiver won’t see it. This is the entire reliability point that the OP was claiming that VHLO partially solves. But it can never be solved with post-acceptance spam classification schemes.
You might argue that it is difficult to arrange to do that work in-line during the SMTP transaction, but that would be incorrect. The sendmail milter mechanism allows one to run arbitrary code of your choice during the SMTP transaction. If the MTA of your choice does not allow that, then perhaps you need a more flexible MTA. My point is that *any* computation that you could do post-acceptance can also be done before you send the answer to the DATA command. Why not be polite and let the SMTP client know the results of that computation?
FUSSP = http://www.rhyolite.com/anti-spam/you-might-be.html
As for running filters at the sending MTA - outbound filtering is widely recommended as a best practice.
However, most spam out there is sent through compromised hosts, botnets etc - that may not even be “regular” smtp servers at all. Or they might relay through the smtp server of their broadband provider etc - which is where the outbound filtering best practice comes in.
Then - if the entire world was following best practices we wouldnt have the sort of spam volumes that we’re seeing. So .. you’re left with inbound filtering.
You dont need VHLO, fancy maps, wild theories from Paul Graham etc (and by the way bayes is not the cureall - doesnt even scale all that well across a distributed mail flow, as I and a few other people had the “pleasure” of telling him back in late 2003 when he was a guest speaker at a SciAm meet the expert type session, at some pub or the other near the MIT campus)
Based on what I see here - I would personally never deploy VHLO with a bargepole.
Spam filtering to clean up a forwarded mail stream is not a bug, its a feature.
——
This memo defines an extension to the SMTP service that provides
protocol support for weak authentication of SMTP clients. Weakly
authenticated clients enjoy an intermediate level of trust: they have
no relying privileges, but can attempt to deliver mail to local
users, are whitelisted from some filters, and may receive DSNs as
needed.
Note that this treatment is what SMTP recommends for all clients.
However, most servers operate filters to limit spam, thereby
affecting the reliability of the mail forwarding system. Verified-
Hello recovers that reliability by providing for uncensored mail
transmission in a framework where authenticated domains are
responsible for the messages they send. In addition, support is
provided for an extensible set of authentication mechanisms, so that
they can be managed and branded.
——-
The ones about senior-ietf-member-* and programmer-*?
What is really difficult about proposals new anti-spam techniques is not the technique but the global modelsthey presume. I think the requirement should be for folks proposing things to present their model in terms of trust, administration and operation in a way that uses no technical references. Tell us how folks (users, operators) would interact and why anyone should believe that this will really happen and what the possible abuse are still possible. What are the assumptions about motives and participation effort?
If the model sounds plausible in terms of that trust and operations overhead, then it might be worth considering the technical details for instantiating it.
/d