Last week, comments were filed with the FCC in response to the Berkman study of international broadband comparisons.

Recall that last month Harvard’s Berkman Center for Internet and Society delivered its draft broadband study (the draft, related post), commissioned by the FCC to assist in the development of a National Broadband Plan through an expert review of literature.

Many of the comments were not supportive of the Harvard Berkman study.

In an earlier blog posting, we had observed that there appeared to be statistical problems in the Berkman study that would not hold up to peer review. Our comments may have understated the extent of the problems.

The Phoenix Centre said that the Berkman study was “so flawed that it cannot be relied upon to formulate public policy”:

the Berkman Study first improperly estimates its econometric model and then incorrectly interprets the results from it. The error in the interpretation is significant. While the [Berkman] Study’s authors verbally conclude that open access policies stimulate increased consumption of broadband, the econometric model they rely upon shows the opposite—open access reduces the consumption of broadband.

The Phoenix Centre also said

[Under the Berkman Study’s analysis] the supply curve is downward sloping! This result implies that as broadband prices rise, network operators supply less broadband. Intuitively, this result makes little sense, violates the law of supply, and muddles interpretation.

Oops.

NTT weighed in to correct errors in Berkman’s analysis of the Japanese market—errors that it said could have been avoided had the authors of the Berkman study interviewed NTT.

First, facilities based competition, not unbundling, has been the key to broadband growth in Japan.

... Second, the report mistates the importance of ‘government-subsidized loans’ to the success of broadband deployment in Japan.

... Third, the Berkman Center’s draft study is internally contradictory.

... In sum, the review of broadband facilities and regulation in Japan contained in the Berkman Center draft study is both factually incorrect and internally inconsistent.

Verizon’s comments said the Berkman study largely ignored the body of literature that demonstrate that unbundling and mandated open access policies have not only failed to improve broadband performance, but frequently have had the opposite effect.

In particular, study after study has shown that government policies such as subsidizing the development of broadband infrastructure and a range of demand-side factors from population density to computer ownership—not the preference for unbundling over intermodal facilities-based competition—are the primary factors explaining the successes of some other countries.

Many of the comments observed that the Berkman study ignored its mandate: to conduct an independent expert review of existing literature and studies about broadband deployment and usage throughout the world.

The New Zealand Institute for the Study of Competition and Regulation, based at Victoria University of Wellington, observed that the Berkman report does not contain an explicit review of the large body of literature on policy and broadband market performance from academic and peer-reviewed sources - fewer than 10 of the more than 400 referenced citations were derived from papers in this body of literature.

The New Zealand submission raised “questions about the standards of competence and integrity applied to this aspect of the Berkman analysis, and by extension to the entire project.”

Johannes Bauer, of Michigan State University’s Department of Telecommunication, Information Studies, and Media was less harsh, but said:

The Commission should take the evidence provided in the study into consideration but be keenly aware of the limitations of the study. Furthermore, caution is appropriate when transferring lessons from abroad to the US. Not only is the market environment different, the institutional, legal, and regulatory environments also differ from those of other countries. Not all observations from abroad can be transferred to national contexts.

NCTA was quoted saying “The Berkman Report, in short, is an advocacy piece, not the work of dispassionate scholarship that the Commission requested.” The Broadcasting & Cable news story said that would make it antithetical to FCC Chairman Julius Genachowski’s stated aim of having data, not ideology, drive conclusions.

(This post originally appeared on our blog.)