I have some reservations about your assertions and conclusions

First off

However, the amount of spam has plummeted from 23,000 in mid 2010 to 5000 now, a drop of over 75%. The contrast couldn’t be starker — spammers are not spamming as much anymore.

Well, the decapitation of several well-known botnets like Coreflood, Mariposa Botnet, Rustock and Bredolab may well mean spammers aren’t spamming as much in the past because they can’t, for the moment. But, we have also seen several huge and sustained spikes on other bots as recently as last week.

It almost looks like the battle against spam is almost over.

Almost, but not quite. I’d be interested in seeing figures from your systems regarding stuff blocked at the network boundaries by other means - IOW stuff that never counted as email, but merely a rejected connect.

Terry, you are speaking from the perspective of a corporate spam filterer (and yours is a perspective limited to ‘what you see’ which may or may not be what others similarly charged are dealing with), which is fine, but what do the ISPs report? How about the huge Freemailers like Yahoo!? We also know that many spammers don’t bother with corporate systems, they don’t even bother with anything else but Yahoo!/Hotmail/Gmail/AOL - nothing ‘in the wild’ to use their term.

Mostly, spammers want to get to Joe Enduser. To do that - they are now spamming Twitter & Facebook, and to a lesser degree LinkIn - does that not count as spam? Yes, closed systems, but just as much - a way to reach their intended target, illicitly.

I’d say the fight is far from over.What you are seeing is a lull in the action, and you may be away from the front. For the moment. It is normally said ironically, but in this case I do mean it guilelessly ‘Good luck with that’.

—
Neil Schwartzman
Executive Director
CAUCE : The Coalition Against Unsolicited Commercial Email

http://cauce.org
http://twitter.com/cauce
IM: caucecanada

TWO articles cross my desktop in less than an hour after seeing this posting

Massive Rise in Malicious Spam - M86
Exponential increase in malware-bearing spam spells trouble for IT
Statistically precise malware campaign promises headaches to the Nth power for IT this month

Neil, in response to your comments:

Well, the decapitation of several well-known botnets like Coreflood, Mariposa Botnet, Rustock and Bredolab may well mean spammers aren’t spamming as much in the past because they can’t, for the moment. But, we have also seen several huge and sustained spikes on other bots as recently as last week.

Coreflood was disabled in April, 2011.  Rustock was disrupted in March, 2011.  Bredolab was disabled in Nov 2010, and Mariposa was disabled in March 2010. The decline in spam began around April 2010, so some of these disruptions are after the major decline.

While it is true that taking botnets offline disrupts spam, it is more often than not a temporary reduction.  The most obvious example is the McColo shutdown in 2008.  Spam plummeted overnight but gradually built back up.  Cutwail was disabled last year with the 3FN takedown but bounced back.

So while I would agree with your point that disrupting botnets can be a temporary takedown, the greater trend is a decline in spam from its peaks rather than a decline-and-subsequent-return-to-normal.

Almost, but not quite. I’d be interested in seeing figures from your systems regarding stuff blocked at the network boundaries by other means - IOW stuff that never counted as email, but merely a rejected connect.

My chart includes rejected connections.

Terry, you are speaking from the perspective of a corporate spam filterer (and yours is a perspective limited to ‘what you see’ which may or may not be what others similarly charged are dealing with), which is fine, but what do the ISPs report? How about the huge Freemailers like Yahoo!?

I agree it’s possible that corporate mail receivers are seeing different trends than free receivers.  I would have to look up Hotmail’s stats, but all of the big receivers of mail who issue threat intelligence reports (Cisco, Symantec) are all reporting declines in email spam so my comments are inline with what the corporate email space is seeing.

To do that - they are now spamming Twitter & Facebook, and to a lesser degree LinkIn - does that not count as spam? Yes, closed systems, but just as much - a way to reach their intended target, illicitly.

I was excluding that type of spam from my post as I was talking about email spam specifically.  This isn’t something I have written about recently but my position is that while email spam has declined, Internet abuse has not. It has merely shifted from one corner of the Internet to the other.  Twitter, Facebook and LinkedIn are the “new” email and that’s where the spam has moved.

Hi Terry,

Perhaps the most important thing to keep in mind is that while the absolute volumes of spam are dropping, it’s mainly because just one particular type of email spam, botnet spam, is declining.

A few years ago, botnet spam was around 95% of all spam.  Infect a few thousand (or 10s of thousands) of machines, and spew away - there were few limits to scale, and the spammer doesn’t have to work very hard.  Over the past few years, the effectiveness of botnet spam has declined because the ability of ISPs to block it has greatly improved to the point that more than 98-99% is blocked, plus the various take downs, means that the return on the investment of botnet development and operation has declined.

Now?  It’s botnet percentage of the total is probably 80%.  A decline from 95% to 80% may not seem like much, but in terms of absolute volumes, once you do the math, the total spam count goes down by a factor of four or more.

Does this mean that spam is ending?  No.  The spam problem is changing.  It’s changed before, and it’s changing again.  It’s adapting to our efforts.  Much is changing to lower volumes of higher-return spam.  Phishes, malware (identity theft, credential theft, DDOS platforms) etc.  We’re seeing marked increases in the amount of spam (often malware and phishes) sent through compromised accounts through real mail servers and so on.

Botnets aren’t dead yet either.  Only a few days ago, Festi reversed most of our volume gains over the past 18 months in a days-long paroxysm of fake anti-virus malware spams.  Hopefully that’s short-lived, but it does indicate that the potential is still there.

The battle isn’t ending, it’s changing shape.  Due to the nature of how it’s changing, I predict rougher times ahead.  It won’t be a cracked pc spewing huge quantities of replica spam that’s easily blocked by IP without adverse consequences to anyone, it’s real mail or web servers spewing smaller amounts of malware, that aren’t as easy to deal with with the current tools - you won’t as easily be able to block a specific IP and not worry about accidentally blocking legitimate content.  Like real viruses, they’re becoming immune to our “antibiotics” and acting differently.

Not to mention ongoing attempts to pollute other Internet services such as Facebook or Twitter.

We’re going to have to keep working hard to stay ahead, and not read too much into the absolute volumes of spam emails.  The volume is lower.  But what’s coming out is much more dangerous.

Ending?  Terry, we’ll talk about that over a beer in 5 years ;-)

Ending? Terry, we’ll talk about that over a beer in 5 years ;-)

Heh. Terry - bring Bill along for those beers. He predicted a similar end some years ago.

I said that spam is down, but the abuse problem is not over.  It’s simply shifting (agree completely with Chris) from “send tons of garbage in email” to diversifying into other things that Chris spoke about, and they are more difficult to block.

The main reason I commented at all is exemplified by your very last comment: “Spam is down, but the abuse problem is not over”.  It’s as if you’re saying that the spam problem is ending, but other forms of abuse aren’t.  They’re not other forms of abuse, they’re _still_ spam…  It’s almost like the old days, when spam went by
open SMTP relays.  When we started to make headway in stopping that, some people did say “spam was defeated”, but it wasn’t, it was just migrating to open web proxies.

They conflated the mechanism with the abuse itself.  Killing open relay didn’t kill spam, it just made it mutate the methods it used.

Only this time, to improve return on investment, not only is spam changing methods, the spam is becoming more dangerous too.

Someone reading without full understanding could take your article to mean that email spam is ending, and it’s okay to relax on that front.

That’s the very last message we want to promote.

I’ll rewrite that quote from you in the way I’d like: “Spam is down, but the spam problem is not over, and rougher times are ahead.”

You see the distinction?

We know what spam is - it’s unsolicited commercial/bulk email (more or less).  I don’t lump in everything as spam and use it as a catch-all for malware, spam, black SEO, drive-by downloading, etc.  I’m going to stick to my guns here because not all types of abuse are the same.

I agree.

But you’re missing the fact that not all kinds of spam are the same.

From a broader perspective, what would you rather have leak through your filters?  A thousand ordinary replica spams, or a hundred Zeus infectors?

If the former came through easily identifiable botnet zombies, and the latter came through shared real mail servers (ie: Yahoo’s or hotmail’s), which is going to be rougher to deal with?  Which battle would you prefer to fight?

I can’t go along with that because it makes the trends meaningless.  If spam volumes were going up, we would say that rougher times are ahead.  But instead spam volumes are going down and therefore… rougher times are ahead.  How can two completely different trends yield the same

The trends are meaningless if you don’t do any further analysis of how the problem is changing.

If you’re in a real battle, and the observer says the number of enemy troops entering the field dropped by a factor of 10, you’d better find out whether the new ones are in tanks, before assuming you’re winning.

You have to have a good idea of what the remaining spam is, and how you’re going to fight it.

For a server provisioning engineer, dropping spam volumes are good.  May not have to buy more machines for example.  But from a broader health-of-the-environment perspective (ie: customers being defrauded, having their identity stolen, losing confidence in the Internet), you have to know the character of the remaining spam, and you also have to take into account what you’re going to need to stop it.

Absolute volumes are dropping.  We all agree on that.  That’s great.

What’s not so great is that the contents of the spam are increasingly malware (identity stealers, keyloggers, zeus infectors etc).  An end-user falling for one of those has far worse consequences for the health-of-the-environment (not to mention the poor end-user) than spending a few bucks on a fake Rolex, which might never come.

What’s also not so great is that the methods of sending them (cracked accounts on real servers, VPS tunnels to shared hosters bought with stolen credit cards etc) is becoming far more prevalent.  What this means for many sites is that they’re going to have to start blocking real servers because they got broken into, but the innocent users trying to use those real servers get blocked too.

In other words, damage to innocent third parties is going to go up.

So, every spam is becoming on average more dangerous, and stopping it will cause more damage to innocent third parties than it has heretofore, because every spam emitter is “holding hostages”.  That’s my definition of rougher.  Dealing with huge bot armies is a lot easier than the fight we’re now beginning to see take shape.

But you’re missing the fact that not all kinds of spam are the same.

I’m not, I just didn’t mention it in my original post.  I am planning on doing a follow up discussing what you’ve mentioned.  I agree with your points that campaigns are narrower and more targeted, and that they are more difficult to detect and remove because they are more integrated into their hosts.

In other words, damage to innocent third parties is going to go up.

Agreed.