The imperatives of measurement seem particularly prominent in today’s social environmental concerns. This is partly because of the problems different societies and international bodies have set themselves to solve. Whether these be the monitoring required to meet climate change targets or prevent biodiversity loss, the formulation of indicators to track progress to meet the targets set for the Sustainable Development Goals, concern about the return of high levels of inequality, or crises in the validity of political polling, how we measure and monitor the world is increasingly on the agenda.
The prevalence of measurement is also partly due to the new possibilities and ways of knowing that are now being revealed through “big data” that track social media records, mobile phone use, and new abundances of machine-read satellite data. These data are themselves complicated constructs; they combine proxies, signals, indicators, and diverse interpretations and inferences. But this does not make them any less “factual”—after all, that is what fact has almost always constituted. The result is that many parts of the world look different now because we have new tools to see them with.
Measuring, monitoring, and counting have come to be close to the heart of research and academia. In Britain, academics’ research is evaluated periodically in national comparative procedures (now called the Research Excellence Framework), and their teaching is about to be also (the Teaching Excellence Framework). Metrics are increasingly likely to dominate these scorings (Wilsden et al. 2016). And independently of any official surveillance, many academics have a fairly good idea as to their citation scores and h-index. In some disciplines, these will even be printed on the back of business cards. The fact that these measures can become such driving goals is a good measure (indicator if you prefer) of the social power of indicators.
Getting measurement right is important because it can become a means of spotting patterns and of holding powers to account. “Without measurement and standards, organizational agents operate under the tyranny of cronyism” (Power 2004: 774). But the means of seeing that measurement affords can also become ways of not seeing, of invisibilizing, of failing to recognize people, places and ways of knowing. As Paige West and colleagues (2006: 254) observed over a decade ago, with reference to a then newly comprehensive database of protected areas, new machines for measurement can become “a way of seeing the world with blind spots and blurred vision not easily perceived by its operators, but these blind spots become darker and fuzzier as the machine becomes better.” Michael Power (2004) argues that when measurement becomes a means of comparing performance in organizations, “new secret organisational worlds” form that create and exploit invisibility in order to appear better in performance metrics.
More than that, we must understand measurement not just as a way of seeing but also as a creative and organizing project that co-produces the very world it aspires to describe. Theodore Porter (1994) is quite clear on this. Understanding measurement (this discipline is called metrology) is not just about how measurement describes society and nature. We have to understand how it reconfigures relations between and with them, and how it imposes and erases meaning. Understanding this is what metrology is all about. As Mark Cooper (2015: 1789) puts it: “it invites us to question the social, political, and scientific conditions under which agreements about measurement and commensuration do or do not occur, and the consequences or effects of particular metrological systems.” Careful historical approaches are required to reconstruct how measurement came to be and what effects it has had (cf. Jacob 2001).
Measurement was integral to colonial and imperial projects. The quest to govern and the quest to measure and classify were intimately related (Freidberg 2007). Histories of the failures and dramas of colonial rule are replete with failed and violent attempts to create new categories of people or nature and ways for these newly identified categories to behave. Similarly, new forms of auditing and rule by standards are likened to new forms of neo-colonial endeavor.
But the transformations of measurement can also be experienced in subtler ways, and closer to home. Porter gives a useful example of the case of the US Corps of Engineers, which was instructed to ensure that the benefits of flood control schemes exceeded the costs. This sounds innocuous enough, except that the means by which costs and benefits were to be calculated was not explicated. Determining the costs and benefits of different proposed schemes then became the stuff of political battles between corporate interests and different branches of the state. More revealing still, when attempts to reconcile competing visions began, quantifying actual costs and benefits proved to be “startlingly elusive” (Porter 1994: 395). Even the agreed format for doing so that eventually came out was still added to and interpreted in idiosyncratic ways. Measurement could not contain the difference it was meant to arbitrate between.
This way of thinking is clearly vital if we are to understand how markets govern, as Cooper (2015) has argued.
To quantify a quality is not merely to solve an intellectual problem. It is to create what Latour calls a center of calculation, surrounded by a network of allies … The quantification of qualities is as much an administrative accomplishment as an intellectual one. And no matter what the skeptics may say, many social qualities have already been successfully quantified, in a variety of ways. Those who seek to do it differently, or to spread the net of quantified qualities still wider, need to consider not only epistemological questions but also moral and political ones. There is strength in numbers, and anyone who proposes to wield them more effectively must ask not only about their validity but also about how the world might be changed by adopting new forms of quantification.
But measurement is not just conjured up through marshaling administrative support; it changes the world because, once established, it also entails altering practices to fit with, or respond to, the measurement. Porter gives the example of the US Forest Service, which had been instructed to cut no more trees than were being renewed through regrowth. Regrowth, however, can be boosted if fertilizers and new varieties of fast-growing tree are introduced—which means that more trees and larger trees can still be harvested (Porter 1994: 401). Indeed, the whole point of incentives offered by diverse forms of neoliberal conservation and environmental policy in the form of payment for environmental services is precisely to change the world by changing the behavior of (rational, profit maximizing) individuals. And some of the individuals can even become rational profit maximizers in the process.
The travesties and distortions required to see the world like a market or state can suggest that managing by counting is inherently flawed. But Power (2004) argues that things are more complicated than that. The social processes that constitute measurement, and how people respond to measurement, create cycles of crisis and reform. He distinguishes between first- and second-order measurement, with the former establishing the classifications that make counting possible and the second combining counts into indices and composite indicators, which can forget the social origins and circumstances that produce them. This can result in multitudes of inappropriate numbers and strange uses of them. And this helps to drive the “cycles of reform” that characterizes measurement. According to Power (2004: 778), the social and political responses to different sorts of (flawed) numbers fluctuate are not about trust or distrust:
Dreams of measurement for control purposes are articulated; these are shown to be defective and/or leading to adverse unintended consequences; new measures and refinements are proposed. Any so-called trust in numbers is tempered by the general cultural acceptance of numbers in all aspects of modern society. Equally, specific episodes of distrust and critique lead to the reconstitution and revision of performance metrics, rather than their abandonment.
Understanding measurement, therefore, is required to understand the societies demanding that measurement, and produced by it. That was the imperative behind this collection. The call for this volume posed a number of challenges. It asked authors to take on a variety of questions, including: How do we approach, measure, quantify, and qualify socio-environmental issues and phenomena? How does what we measure or the way we measure it affect what we know and how we act? How do particular types of, or approaches to, measurement become embedded in epistemic communities and with what consequences? What new things can we learn with new forms and techniques of measurement?
The response was rich, and the eight articles published here capture some of the diversity of interests and approaches. Some of the authors are concerned with how measurement of particular problems can be improved and what approaches might facilitate better counting and accounting. We would group the contributions of Victoria Ramenzoni and David Yoskovitz, Daniel Miller and colleagues, and Eleanor Sterling and colleagues in this category. Other contributions dwelt more on the tensions, flaws, and concealments of measurement. This approach lends itself more to analyses that explore the politics of numbers rather than necessarily suggesting ways of getting better numbers. This describes the work of Pamela McElwee, Amelia Fiske, Jennifer Telesca, David Hoffman, and Scott Schwartz. In detail, the articles are as follows.
Ramenzoni and Yoskovitz explore a practical attempt to measure social impact that took place in the United States. Extreme events such as Hurricane Katrina and the Deepwater Horizon disaster have highlighted how even wealthy states can be ill prepared for extreme events. They need to have better data on aspects of social and community well-being that can be adversely affected by such occasions. Therefore, a number of government agencies were instructed to develop indicators that could capture aspects of social resilience and community adaptation. This review article is based on 15 years of government papers, covering 88 different reports, and reports a number of different shifts in the sort of indicators used to capture community resilience. But in that multitude of works lies part of the problem. For the task of socially embedding these indicators, and creating the required administrative community of measurement that Porter (1994) observed, is complicated by the diversity of administrative communities at work, with competing sets of indicators that they are focusing on. Moreover, the sorts of knowledge mobilized is not necessarily socially sensitive, as so few social scientists have been involved in the agencies concerned.
Miller and colleagues explore the different ways in which the outcomes of forest and biodiversity conservation can be known, which they separate, following Chris Sandbrook and colleagues (2013), into research on forest conservation and management and research for the same. The former, which they label “impact evaluation,” consists in the main (but not exclusively) of quantitative studies of the interaction between conservation interventions and ecological and social well-being. The latter, which they call critical studies, tends to be more place-based, more qualitative, and also more theoretically informed. As Dan Brockington and David Wilkie (2015) have observed, the former tend to have more generalizability, being based on regional-level data that permit comparisons across large areas and sample sizes; the latter more accuracy and intimacy with the particularities of individual cases. Crucially for this collection, impact evaluation is concerned with using indicators, and critical studies is concerned with their construction. But both sets of literature, these authors argue, suffer from a lack of attention to long-term trends. They intriguingly argue for the use of “predictive proxy indicators” as a means of exploring, and predicting, longer-term consequences of interventions in forestry.
Sterling and colleagues take on the task of building locally relevant and meaningful sustainability indicators. Their premise is that international, national, and even regional standards and metrics may fit poorly with local priorities. Indicators cannot just be counted; we must first ask for whom the counting counts. The argument is developed across a series of case studies drawn from a variety of contexts across the Global South, with a particularly strong reference to Pacific Island contexts. Each case study presents a variety of different processes, and results, from attempts to consult and co-create indicators that offer place-based perspectives that are locally accurate. The conclusion of the article offers a means of comparing perspectives derived from the Mauri model, which compares, on a Lickert scale, sustainability in the dimensions of ecosystem, community, family, and indigenous people. These dimensions allow local perspectives and world views to be included, with indicators used that represent physical, cultural, psychological, and spiritual characteristics. The Lickert scale allows commensurability. Somewhat ironically, given the motivation of this article, the Mauri model’s advocates contend it can be used “regardless of community.” But the issue here is not that measurement must be avoided because its simplifications are inherently degrading, but that it can be subverted. It can be used to listen more carefully than normal and be used to speak different truths to power.
McElwee explores the role of measurement and calculation in making ecosystem services (ES) real and how practices of calculation form and travel through networks. Her argument is that the measurement of ecosystem services is currently characterized by its lack of concordance: “actual metrics for ES are diffuse, often incommensurable, and widely contested within the scientific community.” This is partly a problem of conceptual clarity, as defining what constitutes a service (or even the forest providing it) is not agreed. It is partly an empirical challenge—services are hard to measure and proxies abound. Resultant models of ecosystem services can be replete with inaccuracies or estimates, such as using phosphorus runoff as a proxy for all nonpoint water pollution, or estimates of carbon in temperate forests being applied in tropical contexts. The social process of building agreement across epistemic communities is yet to be worked out. This confusion, as McElwee observes, is difficult to reconcile with the “critiques that the very concept of ES may be a fast and slippery slope to commodification of everything in nature.” The process of producing the calculations may well tolerate more dissent, and liberating alternatives, than the critique has thus far allowed.
Fiske examines the case of the Yasuní Ishpingo-Tambococha-Tiputini (Yasuní-ITT) oil concession in Ecuador as a means of understanding how subterranean resources are governed. In a common theme in the collection, she demonstrates that numbers are not revealed or discussed by experts, but are rather used to mediate and translate between normally incalculable benefits and incommensurable goals for nature, society, and the future of both. President Rafael Correa of Ecuador had offered not to drill for oil in this reserve to protect the peoples and nature it contained, if sufficient payment could be raised internationally to compensate for lost revenues. These were not forthcoming, and the subsequent decision to drill was justified both by the revenues it would yield and by the lack of impact it would cause—affecting less than one thousandth of 1 percent of the territory. The empirical basis (if that is a legitimate term in this context) of that claim was never made clear, but then the empirics were not really the point. At stake here were various uncountable goods (unquantified biodiversity, the unknownness of uncontacted peoples), against future worlds of benefit funded by oil revenues—and futures built on further oil extraction. The claim of limited damage became the means by which this prospect could be imagined—and by which it was contested by activists quick to seize upon and reinterpret the political poetry that constituted this “fact.”
Telesca’s concern is the measurement of fisheries, and the simultaneous rise of the science of measuring fish stock, and the collapse and decline of those same stocks. Her argument traces the origins of the term “stock,” drawing on Mary Poovey’s (1998) account of double-entry accounting, and explores the difference between “uncertainty,” whose parameters are hard to know, and “risk,” which can be known, mitigated, and accounted for. Telesca notes that the administrative creation of fish stocks, whose fluctuations could be predicted, and whose uncertainties could be replaced by risks, involves a now distasteful scientific heritage with firm roots in the eugenics movement. Fish “races” had to be identified whose populations, now homogenous, could be predicted. And the consequences have clearly not been good for fish, as Telesca observes: the “outright carnage in fish post–World War II happened despite, or because of, the clever new mathematical models charged with managing uncertainty by quantifying risk under the rationale that they maximized profit.” Models of maximum sustainable yield, combined with legal rights over national waters, encouraged the overfishing of “stocks” whose abundance was assumed until decline (sometimes catastrophic decline) was proven.
Hoffman is concerned with exploring the measurement and understanding of the movement of people around protected areas. The article examines some of the thinking that informs conservation biology and its approach to interactions between people and biodiversity before turning to an old adversary (see debates in Hoffman et al. 2011) in the form of George Wittemyer and colleagues’ (2008) controversial thesis that rural people are migrating to edges of protected areas. This is no longer a particularly credible argument, but Hoffman is keen to bury it more deeply. Perhaps the more important point to emerge from the article is the questioning of the role of meta-studies in conservation. Some insights into general patterns may emerge, but context-specific findings are required to have general policy relevance.
Finally, Schwartz’s remarkable article explores the origins of temperature. Central to his argument is the distinction between heat (warmth) and the measurement and calibration of that energy (which we call temperature). Schwartz is keen to explore the social and economic circumstances that made it likely, and indeed necessary, to fix temperature. He finds his answer in the demands of capitalism to quantify risk and make reality more predictable. Temperature records create trends, and these produce predictive capacity—knowability and knowing. Temperature, it seems, has a history, a set of social circumstances from which precise standardized measurements arose, which is quite separate from the thing it measures.
In toto, there are a number of abiding themes in this collection that are anticipated, and elucidated, in the metrological literature. The first is that measurement is an administrative achievement. The construction of indicators for market governance, community resilience, proxies of performance, heat, fish stocks, and much more requires complicated governing apparatuses and networks of state, private, and civil society interests. A second is that this has unexpected consequences—which perhaps itself should hardly be unexpected at all. The final point is that counting life, whether in societies or environments, is clearly problematic, if not also violent. It does not solve problems but creates a host of new ones.
But it does not follow from the last point that not counting becomes the solution to the problem. It may be a worse fate still not to appear on any register. The issue is not whether to appear, but on whose list, how, for what purpose, and in what circumstances. We cannot avoid measurement. As social beings, we count, calibrate, classify, and measure. How we see ourselves and others, and how we cohere, depends on such processes. It is difficult to imagine societies that do not do any of that. And it is precisely this inevitability and ubiquity of measurement that makes it so necessary to contest it more vigorously.
Brockington, Dan, and David Wilkie. 2015. “Protected Areas and Poverty.” Philosophical Transactions of the Royal Society B 370 (1681): 20140271. doi:10.1098/rstb.2014.0271.
Cooper, Mark. H. 2015. “Measure for Measure? Commensuration, Commodification, and Metrology in Emissions Markets and Beyond.” Environment and Planning A 47: 1787–1804.
Hoffman, David M., Derick Fay, and Lucas Joppa, eds. 2011. “Protected Areas and Migration.” Special section, Conservation and Society 9 (1).
Jacob, Margaret C. 2001. “Factoring Mary Poovey’s A History of the Modern Fact.” History and Theory 40 (2): 280–289.
Poovey, Mary. 1998. A History of the Modern Fact: Problems of Knowledge in the Sciences of Wealth and Society. Chicago: University of Chicago Press.
Power, Michael. 2004. “Counting, Control and Calculation: Reflections on Measuring and Management.” Human Relations 57 (6): 765–783.
Sandbrook, Chris, William M. Adams, Bram Büscher, and Bhaskar Vira. 2013. “Social Research and Biodiversity Conservation.” Conservation Biology 27 (6): 1487–1490. doi:10.1111/cobi.12141.
West, Paige, James Igoe, and Dan Brockington. 2006. “Parks and Peoples: The Social Impact of Protected Areas.” Annual Review of Anthropology 35: 251–277.
Wilsdon, James. 2015. The Metric Tide. The Independent Review of the Role of Metrics in Research Assessment and Management. London, Sage.
Wittemyer, George, Paul Elsen, William T. Bean, A. Coleman O. Burton, and Justin S. Brashares. 2008. “Accelerated Human Population Growth at Protected Area Edges.” Science 321 (5885): 123–126.