Home / Blogs

Blacklisting Under Wrong Assumptions

As an Internet security strategist, I make every effort to understanding just what was installed on our network, and to gain hands-on knowledge. As a result I have installed, tweaked, and worked on Linux, FreeBSD, Mac OS X, Windows XP, Apache, Sendmail, Postfix, and just about anything that the Internet bandwidth can be consumed by. What prompted me to be so hands-on in understanding the guts of operating systems and server software when I could have been relaxing in a summer resort?

The maddening increase of malware and spam. It started as nuisance, until nuisance became threat. And it didn’t stop there. Worms and Trojans that can masquerade as you can potentially damage a hard-earned reputation, especially in Asia Pacific, where reputation and business can be easily lost.

If you analyze the relay of spam- and malware-containing email circulating on the Internet purely through your mail server logs (running the Unix command “tail”), a large proportion seem to come from Asia Pacific hosts, especially those from mainland China. Therefore, many less-experienced systems administrators have simply blocked the access from subnets of Chinese or Asian origin, effectively destroying the fabric of the Internet—messaging. If administrators took pains to analyze these supposedly Asian spam messages by analyzing the full Internet headers, they would have realized that the Asian servers were merely used by the real spammers as open relays, or perhaps as zombie hosts previously infected with the mass mailing worms through the exploitation of operating system vulnerabilities.

I have consistently tried, as a good Internet citizen, to alert the administrators of these vulnerable Asia hosts and open relays of their problems, and I hope that they will patch their systems. But none seem to take my advice too seriously, or at all. So these servers continue to be exploited by many spammers operating out of the Americas, and these Asian hosts are blamed for spamming they did not commit.

If the people running the infected or affected hosts aren’t bothered, why should we care? We may have little choice. These vulnerable hosts give the spammers the ability to mass-mail spam and malware-infested messages worldwide without regards to Internet bandwidth, the clogging of Internet “pipes” worldwide, the slowing down of global infrastructure, and the escalating security breaches and frustration at using the most basic and necessary Internet tool—email. But if these folks don’t care, can we do something?

Short of harassing the governments to take regulatory and legislative action, which regional governments are warming to slowly, we can work more closely with blacklist services that help to provide another venue for pushing host administrators to take prompt action in patching their systems. Some governments, such as those of Singapore and Australia, are more proactive in understanding just why spam is a bad thing, and regulatory and legislative frameworks are incrementally being worked into the system.

Many spam-reporting blacklist services analyze the full mail headers and track down the sending hosts, which are often found to broadband consumer users in North America. One common “culprit” is Comcast.net which, unfortunately for legitimate Comcast users (some are business users), have been blocked out not only by blacklist services, but by corporate network administrators as well. Some administrators even go so far as to not only block the specific spamming IP addresses, but also take the easy route and blocking the entire Comcast.net domain! Such measures are too drastic and draconian, and they spell trouble for legitimate Comcast (and many hundred other hosts) users, especially small and medium-sized businesses trying to make a decent living.

For example, SpamCop‘s reporting service is easy to use, and its response times for unblocking erroneously blocked domains are good. You can report spam to SpamCop by attaching the full spam email and forwarding it or by pasting the entire email into the SpamCop browser-based reporting window. More and more spammers are encoding their spam messages using Base64 (which appears as gibberish unless you are reading in a rich-text-capable email client such as Outlook), and SpamCop can now handle this—using algorithms that decode obfuscated email.

As an experiment, I used a combination of Sendmail access configuration (which lists IP addresses, domains, and email addresses to block or discard) and SpamCop’s DNS blacklist service, I managed to slice spam and malware down by 90%. A collective grassroots movement can make the DNS blacklist more and more sensitive and accurate. Some administrators I chatted with even configured their mail servers to use a whole spectrum of DNS blacklists to create spam blocking “redundancy.” This is because the likes of DNS blacklists are always the target of spammers to bring down, and some have been consistently attacked through denial-of-service (DoS) or even distributed denial-of-service (DDoS) attacks.

Let’s face it: the current antivirus solutions have largely failed. Much as the information released in the U.S. points out that even though 90% of businesses installed antivirus software, 85% of them still have ongoing virus trouble. The situation is a lot worse over in Asia Pacific, because Asia Pacific as a whole lags behind the U.S. in terms of Internet security infrastructure, end-user knowledge, security best practices, and budgets.

Why? Because antivirus software programs are largely reactive, and they respond by attempting to match known historical “patterns.” If a mutant virus does not provide an exact match, the antivirus software will not be smart enough to detect the malicious code and may let it through.

This means we need another layer of protection just before traffic gets to the antivirus gateway. This is easily done by a variety of Procmail or other mail server?compatible scripts (mostly written in the English-like Perl language), to strip out banned attachments altogether. For example, commonly banned attachments such as .bat, .cmd, .com, .dll, .js, .pif, .scr, .shs, and .vbs should be automatically and categorically removed without argument. Some other banned attachments can include multimedia components, which shouldn’t be allowed on limited bandwidth in Asia Pacific, including .mp3, .mov, .mpg, .swf, and so on. A related approach is the use of whitelist attachments, where only certain types of attachment, like .doc, .xls, .ppt, and .pdf, can be allowed through. This is more restrictive, but in certain organizations with limited document types used, this is a possible approach.

While governments take time and prudence to work out regulatory, legislative, and even punitive measures, we as users can surely put in some grassroots-level work and incrementally help to stabilize the Internet infrastructure again, especially for the many who rely on the Internet for survival.

By Dr. Seamus Phan, Entrepreneur & Author

Filed Under


Phil Howard  –  Nov 12, 2003 8:47 PM

I would definitely not consider myself to be a “less-experienced” system administrator as you mention in your article; I have 24 years of such experience, of which the last 10 involves internet access and email servers.  Yet, I do regularly block large chunks of address space on countries like Brazil, China, Korea, Taiwan, and now even India.  The reasons for this is practicality.  Open relays and open proxies are popping up everywhere, and I do block them when I can.  I do use DNS based blacklists as a first line of defense, so my private blocking is limited to what leaks through beyond that.  And I use several for redundancy (minus a couple recently shut down).  I do broadly block a number of address categories, including those that do not have validated reverse DNS, those that are sequentially numbered (generic), and a certain few whole countries (using blackholes.us DNSBLs).

My goal is cutting costs.  So methods that increase processing (cost) are not suitable alternatives.  I try to block based on connecting address (hostname or IP address) as much as possible to mimimize that processing.

I do have one advantage that most of my community of users run Linux, and are not subject to the malware problem.  Those that do run Windows know what they are doing and I leave it up to them to deal with any malware that slips through.  Cutting the cost impact of spam is my primary goal.

I mentioned that I block lots of generic address space.  I am aware that many people run mail servers at home, and might be impacted by this.  But I also believe they have choices in dealing with that.  If they are smart enough, they know the choices and can decide for themselves the best course of action.  I’ll whitelist by IP address.  Or they can just forward to their ISP smart host where that is practical (assuming their ISP doesn’t filter them out for use of their own domain names).

Dr. Seamus Phan  –  Mar 6, 2004 8:03 AM

Hi Phil (Howard), no one is accusing you to be less experienced. I guess you read too much in the wrong direction. But still, what you do may be overly aggressive in the wrong direction, as those spaces you block out are mere victims of spammers who mostly originate from the USA. China, India and many other Asian domains do not yet have the same level of expertise as the USA, or in terms of usable budgets to get more sophisticated equipment and servers. To penalize them too much would be unwise, as China and India are the fastest growing economies, as the larger economies such as USA and Japan have been slowing.

Still, action is better than inaction, although we have to be careful with every little step we take.

Suresh Ramasubramanian  –  Jun 21, 2004 10:39 AM

Seamus, you might want to check out CAUCE Asia Pacific - http://www.apcauce.org

Antispam workshops / conferences every six months, at regional network operators conferences like APRICOT and SANOG.

Speakers like Dave Crocker, Meng Wong, David Harris (the author of pegasus mail / the mercury MTA) etc, plus a “regional update” panel where people from governments, regulatory authorities, ISP associations etc in asiapac regions discuss antispam legislation in their economies.

I guess these conferences are making a small, but significant difference.

If you’d like to attend / present a paper at one of these (the next ones are - Kathmandu, Nepal - July 28-30 2004, and Kyoto, Japan - feb 3rd wk 2005) then email me about it.


Comment Title:

  Notify me of follow-up comments

We encourage you to post comments and engage in discussions that advance this post through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can report it using the link at the end of each comment. Views expressed in the comments do not represent those of CircleID. For more information on our comment policy, see Codes of Conduct.

CircleID Newsletter The Weekly Wrap

More and more professionals are choosing to publish critical posts on CircleID from all corners of the Internet industry. If you find it hard to keep up daily, consider subscribing to our weekly digest. We will provide you a convenient summary report once a week sent directly to your inbox. It's a quick and easy read.

I make a point of reading CircleID. There is no getting around the utility of knowing what thoughtful people are thinking and saying about our industry.

Co-designer of the TCP/IP Protocols & the Architecture of the Internet



Domain Names

Sponsored byVerisign


Sponsored byVerisign

Brand Protection

Sponsored byCSC

Threat Intelligence

Sponsored byWhoisXML API

New TLDs

Sponsored byRadix


Sponsored byDNIB.com

IPv4 Markets

Sponsored byIPv4.Global