The Internet is in for interesting times. Previously, on Renysys’ blog I wrote about the engineering issues and the policy issues facing us over the next five years. But there is at least one large issue still lurking. Most of you will not be surprised to learn that almost all of these issues are outgrowths of a single factor: money. The core of the Internet still doesn’t have a sustainable business model.

Many people are getting rich on the Internet, and almost none of them are spending money to keep the interconnection infrastructure (the “Inter” in “Internet”) growing and expanding. Look at it from a massively oversimplified perspective: Google make their money from the advertising they sell to search audiences. Comcast make their money by offering TV and Internet access on their local cable infrastructure. Amazon makes money selling books and other stuff (including servers and storage space). Most datacenter companies make their money selling space and power inside of their buildings. Spammers make money filling up your inbox with useless crap. Organized crime makes money by launching attacks against profitable companies if they don’t pay extortion. DNS squatters make money registering thousands (or millions) of domain names and sitting on them until someone else is willing to pay. And almost none of this helps the core of the Internet.

Look to the wholesale carriers if you want to see an income statement wasteland. Level 3 lost $1.1b last year. They lost $120m in the most recent quarter alone. Cogent is thrilled because they reported a tiny, tiny positive net income last quarter on top of a yearly loss of $30m in 2007. Global crossing lost $300m in 2007 and $88m in the last quarter they’re reporting, which doesn’t include much of the recent downturn. Other wholesale networks are in the same boat. Dan Golding suggested that it’s more important to look at net cash flows rather than income, but the result is pretty much the same: almost no one is making any money. The only wholesalers who do make money make it on other service offerings: wireless service, metro Ethernet services, VPNs, local phone service, video services and so on. Are there sustainable Internet backbone business models? Does anyone have one?

Just to be clear, the networks I am referring to here are those that make some significant amount of their money by selling to other providers of networking services. They sell in very large capacities to other companies who then sell in smaller capacities and provide support and service and billing. There are very few networks that are pure wholesalers anymore, but to some extent all of the largest networks in the world are wholesalers in at least part of their business. These are networks like Level 3, Sprint, AT&T, Verizon Business, Global Crossing, Abovenet, Savvis, Tiscali and others.

These networks’ business models are in trouble because of price erosion driven by vicious competition. Level 3 and Cogent are routinely blamed for the sharp decline in prices (to levels below $3 per megabit per second in gigabit and above speeds). The first DS3 of Internet that I purchased was $28k/month for 45mb/s of transit, or a whopping $622/Mb/s. This was a new UUNet (AS701) connection that offered a sharp savings off of the $45k/month ANS connection we were replacing (yes, the ANS connection cost $1k/Mb/s). Many people reading this blog paid far more for far slower speeds (because they’re older!).

Low prices are nice and as a consumer of Internet bandwidth I don’t mind them one bit. But the current prices are unsustainable. I don’t say this because I’m some industry shill who wishes prices would go back up (as do many of the financial analysts that I talk to). The proof is in the financials. It does not look like it is possible to run a network of a reasonable size selling bandwidth on a high capacity basis and make money selling full Gig-Es of Internet transit for $3000/month or below.

It’s possible that people who focus on the backbone and on wholesale IP will go away. Edge networks that get value by selling to content providers, selling ads or selling access to consumers will all interconnect directly. And the “backbone provider” will go away. It’s also possible that the backbone provider can make a comeback by developing a business model with some differentiation that might prop up plummeting prices long enough for them to depreciate some equipment and make a little bit of money. It’s not likely, but it’s possible. And I have some thoughts on how it might happen.

The biggest problem that wholesale networks have right now is the complete commoditization of IP service. There is almost nothing, other than price, to differentiate these service providers from each other. Some have bigger brands (AT&T, Sprint), some are more focused on providing Ethernet-based services (Cogent), some are reputed to be more reliable, some are not. But no one has any real data on service provider quality so everyone shops on price. There’s no good reason not to.

Commodity markets are good for most of us. They provide necessary goods at a great price. They provide strong and direct incentives for companies to align investment with market demand, and keep costs down for everyone. Commodity markets tend to misbehave when some number of players can afford to lose money for very long amounts of time. This keeps prices below costs and has the potential to ultimately raise prices (and margins) significantly, provided many competitors go out of business. I believe that this is essentially what several of the low-price leaders in the marketplace are hoping will happen: that they can drive several competitors out of business with unsustainably low costs and afterwards can capture significantly more market share and make money through a combination of economies of scale and raising prices. It’s not a bad strategy provided they have a lot more money than everyone else, or a nearly infinite ability to lose investors’ money on their way to profitability. But it’s risky.

There may be another way forward here: quality differentiation. My colleagues Jim Cowie and Alin Popescu from Renesys have started researching long-term trends in service provider quality, especially with respect to outages and instability. Preliminary results show that networks have significantly different levels of quality with respect to these two metrics. This is based on routing data at the moment, but one can imagine incorporating lots of other, related data. The results are interesting and potentially quite significant.

As transit prices have come down, the real cost and value of one provider or another has shifted. It is not the reliability, responsiveness, features, and service that matter much more than price. If I can buy a gigabit of transit from one network at $5/Mb/s and another for $3, that can save me $2000. But it’s likely that any outage, misconfiguration, or problem with that circuit would cost me much more than $2000. If there were a way to seriously predict which network was likely to be the most reliable over time, or which network would be the easiest to deal with, offer the best response to a denial of service attack, or the lowest-hassle provisioning, those could all get factored into my decision-making. And odds are good that I wouldn’t just blindly pick the cheapest network.

In addition to competing on quality, there’s another way that IP networks may choose to compete: differentiated cost. Right now everyone pays per bit per second thresholded in some way regardless of where those bits come from or are going to. This billing model completely ignores the reality that carrying 1 Mb/s from Los Angeles to Karachi is dramatically more expensive than carrying that same traffic flow to San Jose. People who are carrying cheaper traffic are subsidizing those who have more expensive traffic, because the market has always worked that way and carriers don’t have any good way of accounting for the difference.

But as the cost of Internet transit drops, it has crossed the cost of long-haul transport on many routes. Anyone paying $3/Mb/s for Internet transit that crosses the Pacific is getting a really good deal right now. And the rest of us are paying for it in marginally higher costs. Carriers may try to charge “expensive” customers more for their traffic (or give “cheap” customers even lower prices). This is already done to a very coarse extent on a per-customer basis but carriers may try to do it on a per-connection basis. This would have the huge advantage of aligning revenue with costs in a way that the market does not currently accomplish at all right now. But it would have the huge disadvantage of making the Internet that much more like the phone, with differentiated costs and “long distance” tolls. Flat rate billing with low (or zero) marginal costs creates enormous incentives to experiment and create new services. It’s difficult to imagine what the Internet might look like with those conditions gone.

I don’t pretend to know what is going to happen with the market dynamics of the Internet any more than I know what would happen with the engineering or policy problems that we are facing. But I know for certain that we are in for an interesting 3-5 years as these challenges coincide and we all muddle our way forward.

This post has been reproduced here with kind permission from Renesys.