The rise of the Internet has heralded rapid changes in our society. The opportunities presented by a capable and ubiquitous communications system and a global transportation network have taken some corporations from the multinational to the status of truly global mega-corporation. Good examples of these new corporations include Google, Apple, Amazon, Microsoft and Facebook. There are a handful of large-scale winners in this space and many losers. But this is not the first time we’ve witnessed a period of rapid technological and social change.

The Gilded Age

In 1873, Mark Twain in collaboration with Charles Dudley Warner wrote and published a novel titled “The Gilded Age: A Tale of Today”. The book was remarkable in that its title quickly became synonymous with the concerns of the time relating to graft, materialism, and corruption in public life. The novel’s title is a play on the term “Golden Age,” where the Gilded Age has only a thin layer of overt prosperity, social justice, and political integrity, masking a far baser society that was struggling with poverty, inequality, and injustice. The novel’s satirical treatment of political and social life in Washington are about a period of rapid societal change and the inability of society’s traditional institutions to adjust and come to terms with the rapidly changing social and industrial landscape.

The age to which this novel gave its name was a period of rapid change in the United States. The Civil War in the US was perhaps the first of the major industrial age wars, where the expansion of communications and transportation infrastructure, by way of the telegraph and railways, were complemented by the investment in industrial capacity to raise and support formidable land armies. Once this capacity was in place, the challenge was to divert and repurpose this production capacity at the end of the war. This was the so-called “Reconstruction Era” of 1865–1877. Among other measures at that time was the state-sponsored construction of railroads, which was thought could transform the South out of isolation and endemic poverty. The euphoric response to this injection of public capital into the national economy was, of course, a highly fragile, and the large-scale public spending program all but ensured the ensuing economic crash of 1873.

1901 US cartoon from “Puck” depicting John D. Rockefeller as a powerful monarch (Wikipedia)However, the industrial capacity of the country was undiminished by this crash, and with the assistance of millions of European immigrants, there followed in the 1880’s a period of rapid economic growth, this time using speculative private investment rather than public capital. Wages for the industrial labor force grew by 48% between 1880 and 1890. However, the rewards of all this economic expansion were not well distributed across all parts of American society, so it was also a time of extreme poverty. Not only were some denied any access to this new found industrial wealth, most of this wealth accumulated in the hands of a few. This was a time when a small clique of industrialists could commandeer vast fortunes. This was a time that witnessed the ascent of Cornelius Vanderbilt (railroads and shipping), Andrew Carnegie (steel), James Pierpoint Morgan (banking), Leland Stanford (railroads), Meyer Guggenheim (mining and smelting), and John D. Rockefeller (oil).

This period saw the rise of the corporate entity and as such corporate entities increased in size, so did their ability to concentrate large sums of money, large sets of resources on a single activity. This squeezed out competition from smaller enterprise competition, and lead to comprehensive, well entrenched, monopoly positions, not only for the corporate entity but even more importantly, for their founders. In many cases, it was the rapid exploitation of some novel process or technology that gave the corporate entity its early edge and then it was a more conventional process of simply squeezing out competition. For example, Andrew Carnegie used the Bessemer method to drive down the cost of steel production, and at the same time drive his competitors out of business. J.P. Morgan took this corporation and turned it into US Steel, an industrial behemoth that enlisted a large workforce to work for a single company, with a single corporate purpose. At that time, a big company was big in every dimension. It had a large scale in terms of capital, workforce, geographic points of presence, buildings and plants. Such a massive and rapid accumulation of wealth had a corrupting influence on legislators and undermined the ability of regulatory bodies to function effectively.

The Bosses of the Senate. by J. Ottmann Lith. Co. after Joseph Keppler, Puck (U.S. Senate)

The emerging industrial monopolies took many decades to unravel, and many of these names are still with us as large and influential corporate entities. The period could be summed up as 10 years of explosive competition and innovation, followed by 50 years of coming to terms with the resultant oligarchy!

Parallels with the Internet

What are the parallels with the Internet today? Could we call the current situation the Internet’s Gilded Age?

Within the Internet itself there are strong parallels, although it’s not clear that there has been a single phase of competition followed by agglomeration, but several such phases of competition and agglomeration in various activity sectors over time.

As usual, the antecedents of the Internet’s trajectory extend back into the world of telephony. The initial introduction of the telephone at the World Exposition of 1876 heralded a rush of investment into this new activity, and telephone companies appeared in rapid order. Most were local in scope, and the concept of a single universal service was completely alien. One stood apart, and the Bell company, and under the direction of Howard Vail and with financing from J. P. Morgan, this company aggressively pursued a path of “One System, One Policy” that directly challenged all other operators, and soon prompted an anti-trust action against the Bell company. The arrangement worked out between Howard Vail and the government (the “Kingsbury Commitment”) Bell agreed to divest itself its Western Union telegraph company, leaving the rest of the company in a prime position to further consolidate itself into a massive entrenched monopoly position. Vail saw his company, AT&T, as an enlightened private monopoly undertaking the role of a public utility. In the guise of taking on a commitment to universal service, he was essentially committing to be public interest monopoly that would do no evil, a concept that has been eerily echoed around a century later. At the cost of allowing the most lucrative monopoly in history to flourish, the Kingsbury Commitment enlisted AT&T into a commitment to the goals of a common carriage service in the country.

This was an example of another form of public response to the rise of these corporate giants. The conventional position was then, as now, one that took the view that monopolies are unconditionally an intolerable imposition on the proper functioning of society. The notion of “crony capitalism” and the use of money to purchase political power to further corporate aggrandizement is both a historical and a current problem with large entrenched monopolies. The generic public interest response is to break monopolies up and create in their place a more competitive environment that is not so hostile to new entrants, as happened to Standard Oil in 1909 through the Sherman Act. This was also characterized at the time as siding with labor against capital and acting in the interest of the exploited workers.

A different view and one that motivated the Kingsbury Commitment was to tolerate the monopoly, but at the same time further empower the state, and use the state’s powers to hold these corporate giants to some form of public account. This could be seen as effectively encouraging the monopoly to increase its size even further, but at the same time applying active regulation of the monopoly’s actions to enforce them to work for the public good, or at the very least not to erode this public good any further! In AT&T’s case the model of universal common service using one system, one policy, and one company resonated strongly with a government that was willing to do a deal. The promise of a universally available telephone service for all businesses and citizens alike, all funded through private investment using a single corporate vehicle sounded like an irresistibly good idea. This is despite the observation that AT&T under Vail’s direction, and with J.P. Morgan’s financial support, had in fact taken a highly diverse and competitive industry and literally eradicated all competitors and replaced them with a state-sanctioned monopoly that was then literally too big and too critical to let fail. Many other countries were, understandably, not so trusting of their private sector, and the more common model in other countries was a state enterprise that functioned as a regulated monopoly by virtue of its state ownership.

By the 1970’s the telephone companies were enjoying a form of golden renaissance. The telephone service was largely ubiquitous in the developed world. The completion of digitisation of the underlying network, with the use of time switches and time division multiplexing of the bearer system to complement automated virtual circuit management, allowed the companies to reduce their operating costs with no pressure to reduce their retail prices Being a monopoly meant that there was no competitive market pressure to reduce prices, and being a state enterprise in most cases, the profits were being passed back into the public purse, so that there was no pressure from the state either. Their level of control was close to absolute: they controlled the technology and their suppliers of this technology, the service, the regulatory framework within which they operated.

The seeds of their decline were sewn at the same time. The lucrative operating margins created strong political pressure from potential competitors, particularly in the United States, and the successful anti-trust action in 1984 (after earlier unsuccessful attempts in 1948 and 1956) had far reaching implications. The regional Bell Operating Companies were still local monopolies and still enjoyed considerable margins, yet were unable to directly compete with each other, so they looked offshore to new opportunities. In other countries, there was intense lobbying to open their domestic communications activity to competition, as well as selling the incumbent state-owned telephone company. Governments looked at the potential windfall from the sale of the incumbent telco as a means of alleviating some of the burdens of sovereign debt. Such a move would also relieve governments of the cost of further investment of public sector funds into communications infrastructure. It seemed to be a “win-win” proposition for all concerned. The push for deregulation of the sector assumed almost manic proportions in the late 1980’s when the international development investment banks also took on this agenda. It reached a point of obvious insanity when Fiji, a small island nation in the middle of the Pacific with less than a million inhabitants, was forced to open its communications sector to competition as the price of further developmental funding from the World Bank. Exactly who benefits from having three competitive service providers in this small isolated island nation is somewhat unclear!

The initial response to this global push for structural reform of the telecommunications sector was the rise of the competitive full-service telco, but this was a very tough issue. No-one was going to install a second network of copper wires, so the incumbent was forced to open its networking assets to its competitors by the regulator. Obviously, the newly privatized incumbent saw this as a measure that was hostile to its own revenue base, and the path down deregulation of these full-service copper-based telephone companies was at best tediously slow and dense with obstructionism, lawyers, and protracted legal argument.

The second phase of the deregulation of the industry was to be highly selective as to what the competitive service provider would offer. Mobility was introduced in the 1990’s, and this was immediately a subject of opportunistic competitive offerings. The scale of investment in infrastructure was remarkably small for mobile services, and the startup times short, so this sector saw the rapid entrance of new competitive providers.

There was a second part to this specialization in competition, and that was the offering of these “new” data services, notably the Internet.

For the dial-up ISP of the 1990’s, the upfront costs were remarkably small. The service was an overlay over the telephone network, so there were no changes there. If they terminated the calls, they could even be paid call termination settlement changes from the telco! The users purchased their own computers and modems, so the investment required to set up such an ISP was slight. The Internet exposed new market opportunity in a market that was actively shedding many regulatory constraints via arbitrage of circuit offerings from the entrenched PTT operator. This encouraged the entrance agile, entrepreneurial capital willing to exploit short-term market opportunities exposed through this form of arbitrage. The incumbent volume-based PTT operators unable to redeploy capital and process to meet new demand, and were unwilling to cannibalize existing markets for their older data products. At the same time, the market for Internet services was moving faster than the incumbent telephone companies could react. As one executive from Bell Canada said in 1996, “The pace of new problems appearing is much faster than our ability to solve any of them.”

If this was competitive deregulation of the staid old communications industry, then it certainly enjoyed political favor. At this time, the Internet was the poster child for a new wave of service and innovation heralding new digital wealth. The unbundling of the old vertically integrated monopoly was seen to offer specialized competitive opportunities created in every aspect of service delivery, exploring different access technologies, different technical platforms for services (such as, for example, Skype), different ways of delivering content and services. The evident cost efficiencies of digitization of service delivery, as exemplified by the Internet, exposed other markets to searingly intense competition. The collection of vulnerable industries expanded to include newspapers, greeting cards, music, television, and movies, to name but a few.

The new digital markets arose that complemented the basic Internet Service Provider’s access role. These included digital content providers, web portals and content aggregators, indexing and search engines, sisal networks, and of course advertising. Unbundling of the original vertically integrated full-service model created entirely new sets of industry players, and the resultant waves of euphoria over the past two decades are illustrative of just how broad reaching and disruptive these changes are.

But consumers kept wanting more.

All these digital services require ever higher access speeds, and consumers see these digital services through a lens of the continuous improvements in price and performance that has been sustained by the computer industry rather than from the experience of the far more static (although far more lucrative) static service model that was the bedrock of the telephone industry it had displaced. This promise of continual refinement of the service, delivering more for less was sustained for a short while through process refinements and minor technology tweaks, but ultimately there came a time where the industry needed to achieve economies of scale of operation, and have sufficient scale within each operator to undertake the massive capital investments in new infrastructure. Small and even mid-sized operators were simply not big enough, as happened a little over a century ago in the Gilded Age. The mantra of the Internet enterprise is simple: “Get Big or Get Bought!” Under-capitalised operators simply were not sustainable, and we saw again the rise of large-scale aggregation and bulking in this industry.

The result was much the same as before: a small number of corporate behemoths dominated the Internet and its portfolio of services and content. All others existed essentially at their pleasure and “competition” was once more an exercise in a thin veneer of compliance to outmoded and often irrelevant regulatory structures. For a while it looked like the traditional carriage sector would once more dominate this activity, and while they were no longer state-owned enterprises, the connections between the incumbent large entities and their “home” states still ran deep, and they were still highly influential in the Internet.

However, once more this environment changed.

For a small period, the content and carriage activities existed in largely separate business domains, tied by mutual interdependence. The task of carriage was to carry users to content, which implied that carriage was essential to content. But at the same time, a client/server Internet bereft of servers is useless, so content is essential to carriage. In a world of re-emerging corporate behemoths, such mutual interdependence is unsettling, both to the actors directly involved and to the larger public interest.

The content industry is largely the more lucrative of these two and enjoys far less in the way of regulatory constraint. There is no concept of any universal service obligation or even any effective form of price control in the services they offer. They use internal cross funding that allows them to offer free services to the public, as in free email, free content hosting, free storage, and similar, and fund these services through a second, more occluded, transaction that essentially sells the user’s consumer profile to the highest bidding advertiser. All this happens outside of any significant regulatory constraint and has given the content services industry both considerable wealth and considerable commercial latitude.

It should be no surprise that this industry is now using its capability and capital to eliminate its former dependence on the carriage sector. We are now seeing the rapid rise of the content data network (CDN) model, where instead of an Internet carrying the user to a diverse set of content stores, the content stores are opening local content outlets right next to the user. As all forms of digital services move into CDN hostels, and as the CDN opens outlets that are positioned immediately adjacent to pools of economically valuable consumers, then where does that leave the traditional carriage role in the Internet?

The outlook is not looking all that rosy, and while it may be an early call, it’s likely that we’ve seen the last of the major infrastructure projects being financed by the transit carriers, for the moment at any rate. In many developed Internet consumer markets, there is just no call for end users to access remote services at such massive volumes anymore. Instead, the content providers are moving their mainstream content far closer to the user, and it’s the content distribution systems that are taking the leading role in funding any further expansion in the Internet’s long-distance infrastructure. For the user that means improved service performance and the potential for lower data costs, particularly when considering the carriage cost component. The demands for long distance transit infrastructure are still there, but now it’s a case of the content providers driving the bulk of that demand, and the residual user-driven transit component is becoming an esoteric niche activity that is well outside the mainstream data service. Equally importantly, however, is the observation that these CDNs are essentially private systems, beyond the reach of conventional communications regulatory regimes. They are not based on public communications services, but increasingly use private dark fiber builds, for example.

We are witnessing the waning use of a public communications service model that invests predominately in carriage, such that the user is “transported” to the door of the content bunker. Instead, we are using a model that uses private communications facilities to push content and service towards the user, bypassing most of the previous carriage function. In other words, right now it appears that content is in the ascendant and the carriage folk, particularly the area of long-haul transit carriage, are seeing a marked decline in the perceived value of their services. The mutual independence is being dismantled and the new behemoths of the Internet’s Gilded Age are now content and service operators.

Who are these new barons of today’s Gilded Age? Any casual poll would nominate Google, Facebook, Amazon, Apple and Microsoft. Apple is current the world’s most valuable public company, with a market capitalization of $728B USD. Alphabet, the parent company of Google, has a valuation of $591B USD, and Microsoft weighs in a $499B USD. Collectively, these three companies occupy the top three slots of the ordered list of the most valuable public companies. Amazon and Facebook are also listed in the top 10, while AT&T is notable by its absence on this short list.

But while there are obvious comparisons between these digital barons of today and their counterparts in the late 19th century, there are also some disturbing differences. Previously these corporate giants were giants in every respect. Not only did they have large volumes of capital and resources, but they also employed large numbers or workers. AT&T peaked with some 1 million employees in the 1960’s. US Steel peaked with 340,000 employees. By comparison, Google employs 60,000 people, which is a similar count to that of Apple, while Amazon employs 39,000 individuals, Facebook employ just 17,000 people. Microsoft is by far the largest with 114,000 employees. The obvious change is that these new corporate giants have a far smaller workforce and far shallower social roots. This is an obvious point of concern. These companies have been able to amass massive amounts of capital and encompass relationships with billions of consumers but have done so without reaching deeply into the social fabric of any country. They are in many ways, examples of a virtual corporate entity that exists more in the digital ether, rather than with bricks, mortar, and workers. And, in a similar manner, their taxation bill sometimes appears to also enjoy a similar degree of virtuality. In a regulatory sense, they also enjoy considerable freedom. These are not carriage enterprises and would not normally fall into the communications sector in terms of regulatory oversight.

In today’s Internet, what do we mean in a policy sense by concepts such as “universal service obligation” “network neutrality” “rights of access” or even “market dominance” when we are talking about diverse CDNs as being the dominant actors in the Internet ecosystem? At some point in the past few years or so the dominant position across the entire Internet has been occupied by a very small number of players who are moving far faster than the regulatory framework. These actors have such market dominance that they can set their own rules of engagement with the consumers of their services their methods of engagement with each other, third party suppliers, regulators and of course governments. By taking up an agile position with these emergent technologies, these players have been able to amass vast fortunes, with little in the way of accountability to a broader common public good. These are today’s overreaching oligarchies.

The oligarchies of the gilded age were highly persistent. JPMorgan Chase is still among today’s top 10 corporates with a market capitalization of $308B USD, as is General Electric ($279B USD), and a by-product of Standard Oil, Exxon Mobil, is also in this exclusive club ($4347B USD). The lesson appears to be that once these corporate monoliths have been created, the effort of curbing their excessive and overbearing influence can take us many decades to dismantle.

Why would we even want to do so? Surely this is a time of rapid social enrichment? Like the original Gilded Age aren’t we all riding on a rising tide of affluence? It seems not. As in the past, there remains large-scale endemic poverty and massive inequities in our world. As the demands of our post-industrial workforce change, we are seeing increasing numbers of displaced workers. This economy is not based on large unskilled or even skilled workforces, and even the skill sets that are in demand are highly specialized. In the same manner that the industrial age displaced the rural cottage industries, we see another wave of change. But this new world is not for everyone, and it’s not that the jobs are in different factories this time. Many of these jobs have simply disappeared, there is an increasing pool of people who are highly reliant on social welfare. But social welfare demands social funding, and we return once more to the observation that these new corporate giants have essentially tiny workforces, and are not deeply rooted in any particular national social fabric. Highly selective corporate philanthropy appears to be a rather poor substitute for conventional corporate taxation.

In today’s world, there is once more a massive concentration of corporate power. There are fewer public companies, and more are disappearing than appearing while this process of concentration and aggregation is underway. Now corporatism is the new thinking, and any politician that has a desire to curb corporate influence and social power is forced to work with a sparse set of thin regulatory frameworks to advance the cause of the public good. The more common mantra is to surrender the ground to so-called industry self-regulation, and hope that the resultant feeding frenzy does not generate an unpalatable level of collateral damage.

There are very few times and very few industries where the private sector has been entirely capable of looking after the public interest. “Do No Evil” is a hopelessly inadequate replacement for the traditionally regulated frameworks that provide our society with essential public services. We now need robust active public regulatory frameworks that can support vibrant industry competition, fundamental innovation and maintain the enduring public value of our Internet.

This is no longer just a conversation about carriage and communications. It is probably not even a conversation about carriage and communications at all. The changing face of the Internet is no longer a matter of public communications, but a matter of public services. And with this observation, we are back to a more basic theme. The essential topic of the conversation is how to strike a sustainable balance between a rapacious private sector that has amassed overarching control of the digital service and content space, and the needs of the larger society in which we all would like some equity of opportunity to thrive and benefit from the outcomes of this new digital age.

We are indeed living in the Internet’s Gilded Age.