Two of us from the “LSE Tech” team attended the latest in a series of invited workshops held at the leading organization for the analysis of the internet, the Cooperative Association for Internet Data Analysis (CAIDA) at the University of California, San Diego. We met with a well-established group who have honed their techniques for analyzing various forms of metrics of the internet. In recent years they have begun to take a refreshingly honest view of the gap between their established approaches and the needs of economists and other social scientists to measure and assess different characteristics.
There remain many inhibitors to bridging the gaps between the engineering approaches that have dominated and the social science questions that are emerging. Some of those are inherent to engineering approaches to the problem and in particular the persistence of an elegant, efficient and logical way of describing the layered model of the internet despite the clear evidence that neither internet businesses nor users behave as if that model describes what they are doing. The underlying problem this creates is that inappropriate emphasis is placed on certain kinds of measurements. Some of that comes about for the obvious reason that analysts will measure what is readily measurable. The knock-on effects, however, are extensive and much policy is made based on concepts of divisions between layers and much of the commercial debate about issues such as net neutrality is unnecessarily restricted. Those restrictions seem appropriate when considering only the US structures and dynamics, and the outmoded layered model of the internet. They also focus attention on end-users at the expense of efforts to understand many other aspects of network functions.
In addition to the sources of gaps associated with engineering imperatives, the discussion made it apparent that we social scientists are, not unexpectedly, the source of confusion stemming from our inability to articulate well enough the problems we wish to solve, the metrics we wish to have, and the analytical techniques we wish to use. Nevertheless, the discussions were extremely helpful in pushing that agenda further. In some of the formal presentations and in much of the debate and discussion it became more clear what kinds of measures would further social science enquiry. While some of these might be derived with little effort from existing metrics, and others can be acquired through additional questions on national census forms, new ideas are required to bring this agenda further.
In particular, measures relevant to the analysis of competition are at the core of studies from legal, regulatory as well as economic perspectives. While some work, such as that of Eli Noam on concentration in the media sector, provides the basis of parts of the analysis, much more is required to satisfy a variety of economic enquiries about industry structure and competition. An especially valuable discussion centred on ways to conceptualize and measure value added in the digital economy. The prevailing models from manufacturing industry and the services sector are inappropriate for digital goods and services. While the some indicators are easily measurable, such as revenues from internet services, it is much more difficult to gather evidence about value creation. For this reason, we cannot lend much credence to any existing estimate of the contribution of the internet to national economies. Rectifying this should become a high priority for the economics community as well as national and international census offices.
One area that has attracted much attention in recent years is measures of quality of experience [QoE]. While a more commercially, politically and socially meaningful category of measurement than quality of service or simply broadband speed, it is not as satisfactory an indicator of economic activity as some seem to expect. While clearly important, this is an area where the significance of the main metrics is especially ephemeral and only indirectly related to firm behaviour in the design of business models. Nevertheless, the discussion and new approaches to QoE does serve to shift attention away from some traditional metrics such as speed to those such as latency, jitter, outages, and other factors that relate to customer retention and some aspects of usage behaviour.
In summary, we found that many features internet are already counted, but not all of it is relevant, even to the questions that initially sparked the data gathering exercises long ago. Rather few data sets are gathered that are useful for economic analyses, especially for time series analyses and in ways that allow for disaggregation. Nothing yet addresses the dramatic disconnect between revenue and volume for different kinds of traffic, a key but poorly understood element of the net neutrality debate. While we are better able to recognize the recurring problems of incompatible measures, categories of measures, and concepts of measures, there is more that the academic community needs to do to solve these problems. And while the commercial participants appreciate and understand the details of these problems of counting, they do not yet share a sense of the difficulties that inadequate data and the existing information asymmetries create for analysts or for the policy and legal community.
**This post is linked to the briefing on the WIE 2012 Workshop ellaborated by Silvia Elaluf-Calderwood**
——
Dr Jonathan Liebenau is a Reader in Technology Management in the LSE Department of Management. He focuses on information and communication technology in the economy. He has worked in academic administration, technology policy, and the economic history of science-based industry and is the author or editor of several books and over 70 other major publications and has provided research and consultancy services to companies and government agencies, including: Dell, BT, IBM, Microsoft, Tata Consultancy Services, Nortel, Lloyd Thompson, Nokia, ETNO and to the UK Office of Science & Innovation, the Department of Business Innovation & Skills, and the Home Office.
We welcome your comments! Take a look at our comments policy and send us your comments on this post
